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Optimal sampled-data control, and generalizations on time 

scales 
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Abstract 

In this paper, we derive a version of the Pontryagin maximum principle for general finite¬ 
dimensional nonlinear optimal sampled-data control problems. Our framework is actually 
much more general, and we treat optimal control problems for which the state variable evolves 
on a given time scale (arbitrary non-empty closed subset of R), and the control variable evolves 
on a smaller time scale. Sampled-data systems are then a particular case. Our proof is based on 
the construction of appropriate needle-like variations and on the Ekeland variational principle. 
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1 Introduction 

Optimal control theory is concerned with the analysis of controlled dynamical systems, where one 
aims at steering such a system from a given configuration to some desired target by minimizing 
some criterion. The Pontryagin maximum principle (in short, PMP), established at the end of the 
50’s for general finite-dimensional nonlinear continuous-time dynamics (see [46], and see [30] for 
the history of this discovery), is certainly the milestone of the classical optimal control theory. It 
provides a first-order necessary condition for optimality, by asserting that any optimal trajectory 
must be the projection of an extremal. The PMP then reduces the search of optimal trajectories 
to a boundary value problem posed on extremals. Optimal control theory, and in particular the 
PMP, has an immense field of applications in various domains, and it is not our aim here to list 
them. 

We speak of a purely continuous-time optimal control problem, when both the state q and the 
control u evolve continuously in time, and the control system under consideration has the form 

q{t) = f{t,q{t),u{t)), for a.e. t G M'*', 

where q{t) G ffi" and u{t) G 17 C K”^. Such models assume that the control is permanent, that 
is, the value of u{t) can be chosen at each time t G M"''. We refer the reader to textbooks on 
continuous optimal control theory such as ]4, 13, 14, 18, 20, 21, 33, 42, 43, 46, 47, 49, 50] for many 
examples of theoretical or practical applications. 
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We speak of a purely discrete-time optimal control problem, when both the state q and the 
control u evolve in a discrete way in time, and the control system under consideration has the form 

qk+i-qk = f{k,qk,Uk), A: G N, 

where qk G M" and Uk G ^ C K™. As in the continuous case, such models assume that the 
control is permanent, that is, the value of Uk can be chosen at each time fc G N. A version of the 
PMP for such discrete-time control systems has been established in [32, 39, 41] under appropriate 
convexity assumptions. The considerable development of the discrete-time control theory was 
in particular motivated by the need of considering digital systems or discrete approximations in 
numerical simulations of differential control systems (see the textbooks [12, 24, 45, 49]). It can 
be noted that some early works devoted to the discrete-time PMP (like [27]) are mathematically 
incorrect. Some counterexamples were provided in [12] (see also [45]), showing that, as is now well 
known, the exact analogous of the continuous-time PMP does not hold at the discrete level. More 
precisely, the maximization condition of the continuous-time PMP cannot be expected to hold in 
general in the discrete-time case. Nevertheless, a weaker condition can be derived, in terms of 
nonpositive gradient condition (see [12, Theorem 42.1]). 

We speak of an optimal sampled-data control problem, when the state q evolves continuously 
in time, whereas the control u evolves in a discrete way in time. This hybrid situation is often 
considered in practice for problems in which the evolution of the state is very quick (and thus 
can be considered continuous) with respect to that of the control. We often speak, in that case, 
of digital control. This refers to a situation where, due for instance to hardware limitations or to 
technical difficulties, the value u{t) of the control can be chosen only at times t = kT, where T > 0 
is fixed and fc G N. This means that, once the value uikT) is fixed, u{t) remains constant over the 
time interval [kT, (fc -|- 1)T). Hence the trajectory q evolves according to 

q{t) = f{t, q{t),u{kT)), for a.e. t G [kT, (fc -|- l)r), fc G N. 

In other words, this sample-and-hold procedure consists of “freezing” the value of u at each con¬ 
trolling time t = kT on the corresponding sampling time interval [kT, (fc -|- 1)T), where T is called 
the sampling period. In this situation, the control of the system is clearly nonpermanent. 

To the best of our knowledge, the classical optimal control theory does not treat general nonlin¬ 
ear optimal sampled-data control problems, but concerns either purely continuous-time, or purely 
discrete-time optimal control problems. It is one of our objectives to derive, in this paper, a PMP 
which can be applied to general nonlinear optimal sampled-data control problems. 

Actually, we will be able to establish a PMP in the much more general framework of time 
scales, which unifies and extends continuous-time and discrete-time issues. But, before coming to 
that point, we feel that it is of interest to enunciate a PMP in the particular case of sampled-data 
systems and where the set of pointwise constraints on the controls is convex. 


PMP for optimal sampled-data control problems and fl convex. Let n, m and j be 

nonzero integers. Let T > 0 be an arbitrary sampling period. In what follows, for any real number 
t, we denote by E{t) the integer part of t, defined as the unique integer such that E{t) <t< E{t)-\-l. 
Note that fc = E{t/T) whenever kT < t < (k -\- 1)T. We consider the general nonlinear optimal 
sampled-data control problem 


(OSDCP)“y 


min f {T,q{T),u{k{T)T)) dr, with fc(T) = A(t/T), 

Jo 

< q{t) = fit, qif), ■u(fc(t)T)), with kit) = Eit/T), 
u{kT) G H, 

^5(9(0).^(A/)) e S. 
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Here, / : K x M” x K™ —)• K" and /° : K x M” x —)• M are continuous, and of class in {q, u), 
g : K" X R" —>■ is of class C^, and H (resp., S) is a non-empty closed convex subset of R™ (resp., 
of R-l). The final time t/ > 0 can be fixed or not. 

Note that, under appropriate (usual) compactness and convexity assumptions, the optimal 
control problem (OSDCP)^^ has at least one solution (see Theorem 2 in Section 2.2). 

Recall that g is said to be submersive at a point (( 71 ,( 72 ) G R" x K" if the differential of g at 
this point is surjective. We define the Hamiltonian i? : R x R" x R" x R x R™ —>■ R, as usual, by 
Hit,q,p,p°,u) = {p,f{t,q,u))Mr. + p° f°{t,q,u). 

Theorem 1 (Pontryagin maximum principle for (OSDCP)^j,). If a trajectory q*, defined on 
[0,tj] and associated with a sampled-data control u*, is an optimal solution 0 /(OSDCP)*^,, then 
there exists a nontrivial couple {p,p^), where p : [0,tj] —>■ R" is an absolutely continuous mapping 
(called adjoint vector) and p^ < 0, such that the following conditions hold: 

• Extremal equations: 

f)TJ f)TJ 

q*{t) = -^it,q*{t),p{t),p°,u*{k{t)T)), p{t) = - — {t,q*{t),p{t),p°,u*{k{t)T)), 

for almost every t € [0,tp, with k(t) = E{t/T). 

• Maximization condition: 

Eor every controlling time kT S [0,t)-) such that {k + 1)T < t), we have 


r{k+l)T Qjj 
IkT du 


{T,q*{T),p{T),p°,u*{kT)) dr , y-u*{kT)) < 0, 


( 1 ) 


for every y G In the case where kT G [0,t*j:) with (k + 1)T > t*p the above maximization 
condition is still valid provided (fc -|- 1)T is replaced with t*j. 

• Transversality conditions on the adjoint vector: 

If g is submersive at {q*{0), q*{t*j:)), then the nontrivial couple (jp,p^) can be selected to satisfy 

P(0) = - (^(9*(0),g*(t/))^ P(^/) = (^^(9*(0)’9*(i/))^ -fp, 

where —ip belongs to the orthogonal o/S at the point g{q*{0),q*{t*j:)) G S. 

• Transversality condition on the final time: 

If the final time is left free in the optimal control problem (OSDCP)*^,, f/1^ >0 and if f 
and /° are of class with respect to t in a neighborhood oft), then the nontrivial couple 
(p,p°) can he moreover selected to satisfy 

H{t),q*{t)),p{t)),p\u*{k)T)) = Q, 

where k) = E{t)/T) whenever t) ^ NT, and k) = E{t)/T) — 1 whenever t) G NT. 

Note that the only difference with the usual statement of the PMP for purely continuous¬ 
time optimal control problems is in the maximization condition. Here, for sampled-data control 
systems, the usual pointwise maximization condition of the Hamiltonian is replaced with the in¬ 
equality (1). This is not a surprise, because already in the purely discrete case, as mentioned 
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earlier, the pointwise maximization condition fails to be true in general, and must be replaced with 
a weaker condition. 

The condition (1), which is satisfied for every y S n, gives a necessary condition allowing to 
compute u*{kT) in general, and this, for all controlling times kT G [0,tp. We will provide in 
Section 3.1a simple optimal consumption problem with sampled-data control, and show how these 
computations can be done in a simple way. 

Note that the optimal sampled-data control problem (OSDCP)*^, can of course be seen as 
a finite-dimensional optimization problem where the unknowns are u*{kT), with k G N such 
that kT G [Ojtf). The same remark holds, by the way, for purely discrete-time optimal control 
problems. One could then apply classical Lagrange multiplier (or KKT) rules to such optimization 
problems with constraints (numerically, this leads to direct methods). The Pontryagin maximum 
principle is a far-reaching version of the Lagrange multiplier rule, yielding more precise information 
and reducing the initial optimal control problem to a shooting problem (see, e.g., [51] for such a 
discussion). 

Extension to the time scale framework. In this paper, we actually establish a version of 
Theorem 1 in a much more general framework, allowing for example to study sampled-data control 
systems where the control can be permanent on a first time interval, then sampled on a finite set, 
then permanent again, etc. More precisely. Theorem 1 can be extended to a general framework 
in which the set of controlling times is not NT but some arbitrary non-empty closed subset of K 
(i.e., a time scale), and also the state may evolve on another time scale. We will state a PMP for 
such general systems in Section 2.3 (see Theorem 3). Since such systems, where we have two time 
scales (one for the state and one for the control), can be viewed as a generalization of sampled-data 
control systems, we will refer to them as sampled-data control systems on time scales. 

Let us first recall and motivate the notion of time scale. The time scale theory was introduced 
in [34] in order to unify discrete and continuous analysis. By definition, a time scale T is an 
arbitrary non-empty closed subset of K, and a dynamical system is said to be posed on the time 
scale T whenever the time variable evolves along this set T. The time scale theory aims at closing 
the gap between continuous and discrete cases, and allows one to treat general processes involving 
both continuous-time and discrete-time variables. The purely continuous-time case corresponds to 
T = IR+ and the purely discrete-time case corresponds to T = N. But a time scale can be much 
more general (see, e.g., [29, 44] for a study of a seasonally breeding population whose generations 
do not overlap, and see [6] for applications to economics), and can even be a Cantor set. Many 
notions of standard calculus have been extended to the time scale framework, and we refer the 
reader to [1, 2, 10, 11] for details on that theory. 

The theory of the calculus of variations on time scales, initiated in [8], has been well studied 
in the existing literature (see, e.g., [7, 9, 17, 28, 35, 38]). In [36, 37], the authors establish a 
weak version of the PMP (with a nonpositive gradient condition) for control systems defined on 
general time scales. In [16], we derived a strong version of the PMP, in a very general time scale 
setting, encompassing both the purely continuous-time PMP (with a maximization condition) and 
the purely discrete-time PMP (with a nonpositive gradient condition). 

All these works are concerned with control systems defined on general time scales with per¬ 
manent control. The main objective of the present paper is to handle control systems defined on 
general time scales with nonpermanent control, that we refer to as sampled-data control systems 
on time scales, and for which we assume that the state and the control are allowed to evolve on 
different time scales (the time scale of the control being a subset of the time scale of the state). 
This framework is the natural extension of the classical sampled-data setting, and allows to treat 
simultaneously many sampling-data control situations. 

Our main result is a PMP for general finite-dimensional nonlinear optimal sampled-data control 
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problems on time scales. Note that our result will be derived without any convexity assumption 
on the set fl of pointwise constraints on the controls. Our proof is based on the construction 
of appropriate needle-like variations and on the Ekeland variational principle. In the case of a 
permanent control, our statement encompasses the time scale version of the PMP obtained in [IG], 
and a fortiori it also encompasses the classical continuous and discrete versions of the PMP. 

Organization of the paper. In Section 2, after having recalled several basic facts in time scale 
calculus, we define a general nonlinear optimal sampled-data control problem defined on time 
scales, and we state a Pontryagin maximum principle (Theorem 3) for such problems. Section 3 
is devoted to some applications of Theorem 3 and further comments. Section 4 is devoted to the 
proof of Theorem 3. 


2 Main result 

Let T be a time scale, that is, an arbitrary non-empty closed subset of K. Without loss of generality, 
we assume that T is bounded below, denoting by a = minT, and unbounded above. ^ Throughout 
the paper, T will be the time scale on which the state of the control system evolves. 

We start the section by recalling some useful notations and basic results of time scale calculus, 
in particular the notion of Lebesgue A-measure and of absolutely continuous function within the 
time scale setting. The reader already acquainted with time scale calculus may jump directly to 
Section 2.2. 


2.1 Preliminaries on time scale calculus 

The forward jump operator cr : T —)■ T is defined by a{t) = inf{s S T | s > t} for every t S T. A 
point t € T is said to be right-scattered whenever a{t) > t. A point t S T is said to be right-dense 
whenever a{t) = t. We denote by RS the set of all right-scattered points of T, and by RD the set 
of all right-dense points of T. Note that RS is at most countable (see [23, Lemma 3.1]) and that 
RD is the complement of RS in T. The graininess function p :T —>■ K’*' is defined by p{t) = a{t) — t 
for every t G T. 

For every subset A of M, we denote by Aj = A n T. An interval of T is defined by Ij where I 
is an interval of M. For every b G T\{a} and every s G [a, 6 )t H RD, we set 

V^’' = {/3>0|s + /3e [s,6]t}. (2) 

Note that 0 is not isolated in V®’^. 


A-differentiability. Let n S N*. The notations || • Hr™ and (•, ■)Rr^ respectively stand for the usual 
Euclidean norm and scalar product of K". A function q : T —>■ K" is said to be A-differentiable at 
t S T if the limit 


q^{t) = hm 

s—yt 

sGT 


QHt)-qis) 
a{t) — s 


exists in K”, where q'^ = qo a. Recall that, if t G RD, then q is A-differentiable at t if and only if 
the limit of as s —>■ t, s G T, exists; in that case it is equal to q^(t). If t G RS and if q is 

continuous at t, then q is A-differentiable at t, and q^{t) = ‘‘ [10] )• 


^In this paper we only work on a bounded subinterval of type [a,b] fl T with a, b G T. It is not restrictive to 
assume that a = minT and that T is unbounded above. On the other hand, these two assumptions widely simplify 
the notations introduced in Section 2.1 (otherwise we would have, in all further statements, to distinguish between 
points of T\{maxT} and maxT). 
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If g, g' : T —)• K" are both A-differentiable at t G T, then the scalar product (g, is 
A-differentiable at t and 

(g,9')K"W = (9^W>9 "'W)r" + {q{t),Q'^it))R^ = {q^{t),q\t))R^ + (3) 

These equalities are usually called Leibniz formulas (see [10, Theorem 1.20]). 

Lebesgue A-measure and Lebesgue A-integrability. Let /ta be the Lebesgue A-measure 
on T defined in terms of Caratheodory extension in [11, Chapter 5]. We also refer the reader to 
[3, 5, 23, 31] for more details on the /^A-measure theory. For all (c, d) G such that c < d, one 
has haHc, ^(t) = d — c. Recall that ^ C T is a ^A-measurable set of T if and only if A is an usual 
/iL-measurable set of K, where denotes the usual Lebesgue measure (see [23, Proposition 3.1]). 
Moreover, if A C T, then 

Ha{A) = fj.L{A) + fi{r). 

rGAnRS 

Let A C T. A property is said to hold A-almost everywhere (in short, A-a.e.) on A if it holds 
for every t G A\A', where A' C A is some /r a- measurable subset of T satisfying ha{A') = 0. In 
particular, since /.ja({?■}) = m(?’) > 0 for every r G RS, we conclude that, if a property holds A-a.e. 
on A, then it holds for every r G A n RS. 

Similarly, if A C T is such that ^a{A) = 0, then A C RD. 

Let n G N* and let A C T be a ^A-measurable subset of T. Consider a function q defined 
A-a.e. on A with values in K”. Let A = AU (r, cr(r))reAnRS) and let q be the extension of q defined 
/Ti-a.e. on A by q(t) — q(t) whenever t G A, and by q{t) — q[r) whenever t G (r, cr(r)), for every 
r G A n RS. Recall that q is /rA-measurable on A if and only if q is /tl- measurable on A (see [23, 
Proposition 4.1]). 

Let n G N* and let A C T be a ^A-measurable subset of T. The functional space L|P(A,K") 
is the set of all functions q defined A-a.e. on A, with values in K", that are /rA-measurable on A 
and bounded A-almost everywhere. Endowed with the norm 1|911 l”(a,R") = supess,-g^ lk(''‘)||R'*, 
it is a Banach space (see [3, Theorem 2.5]). The functional space L^(A,IR") is the set of all 
functions q defined A-a.e. on A, with values in K", that are /rA-measurable on A and such that 
||g(r)||Rn. Ar < -1-00. Endowed with the norm ||(z||lRa,K") = X 4 lk('^)l|K" At, it is a Banach 
space (see [3, Theorem 2.5]). We recall here that if g G L]j.(A,K") then 

g(r)dT= / g(r)dr+ Y d'{r)q{r), 

rGAnRS 

see [23, Theorems 5.1 and 5.2]. Note that if A is bounded then L(]?(A,K") C L]j.(A,K"). 

Absolutely continuous functions. Let n G N* and let (c, d) G such that c < d. Let 
C([c, (i]T, K") denote the space of continuous functions defined on [c, d]j with values in K”. Endowed 
with its usual uniform norm jj • jjoo, it is a Banach space. Let AC([c, (i]T,K") denote the subspace 
of absolutely continuous functions. 

Let to G [c,d]f and q : [c, d]T —t K"- It is easily derived from [22, Theorem 4.1] that q G 
AC([c, d]f, K") if and only if q is A-differentiable A-a.e. on [c, d)T and satisfies G L][.([c, d)T, R"), 
and for every t G [c, (i]T one has q{t) = g(to) + g^(r) Ar whenever t > to, and q{t) = 

- I[t,toh whenever t < to- 

Assume that q G L][.([c, ci)T,R"), and let Q be the function defined on [c,d]^ by Q{t) = 
(7(r) At whenever t > tg, and by Q{t) = — (7(t) At whenever t < tg. Then Q G 

AC([c, (i]T) and — q A-a.e. on [c, d)T. 
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Note that, if g G AC([c, d]T, R") is such that = 0 A-a.e. on [c, ci)T, then q is constant on 
[c, fi]T, and that, if q, q' G AC([c, d]T,R"), then {q, g')®" G AC([c, cZJt, R) and the Leibniz formula ( 3 ) 
is available A-a.e. on [c, (i)T. 

For every q G L^([c,ci)T,R"), we denote by ^[c^d)r{Q) the set of points t G [c,d)T that are 
A-Lebesgue points of q. It holds ^A{^[c,d)ji<l)) = Ma([c, d)T) = d — c, and 


lim 

/3^0 


1 


,s+/3)t 


q{T) At = q{s), 


for every s G ( 9 ) H RD, where is defined by (2). 


2.2 Optimal sampled-data control problems on time scales 

Let Ti be another time scale. Throughout the paper, Ti will be the time scale on which the control 
evolves. We assume that Ti c T.^ 

Similarly to T, we assume that minTi = a and that Ti is unbounded above. As in the previous 
paragraph, we introduce the notations ui, RSi, RDi, Ai, etc., associated with the time scale 
Ti. Since Ti C T, note that RS C RSi and RDi C RD. We define the map 


$ : 


T —Ti 

t I—S' $(t) = sup {s G Ti I s < t}. 


For every t G Ti, we have $(t) = t. For every t G T\Ti, we have ^{t) G RSi and ^{t) < t < 
cri($(t)). Note that, if t G T is such that $(t) G RDi, then t G Ti. 

In what follows, given a function u : Ti —)■ R, we denote by the composition u o $ : T —)■ R. 
Of course, when dealing with functions having multiple components, this composition is applied 
to each component. Let us mention, at this step, that if it G L^JP(Ti,R) then it'^ G L“(T,R) (see 
Proposition 1 and more properties in Section 4.1.1). 

Let n, m and j be nonzero integers. We consider the general nonlinear optimal sampled-data 
control problem on time scales 


(OSDCP)t^ 


min f /°(r,g(r),M'^(r)) Ar, 
d[a,b)r 

< q^{t) = 

€ LtJ 

.5(9(0), iW) G S. 


( 4 ) 


Here, the trajectory of the system is g : T —>• R", the mappings / : T x R” x R™ —>• R" and 
/° : T X R" X R™ —>• R are continuous, of class in (q, it), the mapping g : R” x R" — R-’ is of 
class C^, n is a non-empty closed subset of R”^, and S is a non-empty closed convex subset of R-^ . 
The final time 6 G T can be fixed or not. 


Remark 1. We recall that, given it G L|P(Ti,R'"), we say that g is a solution of (4) on If if: 

1. It is an interval of T satisfying a € If and /T\{a} ^ 0; 

^Indeed, it is not natural to consider controlling times t £ Ti at which the dynamics does not evolve, that is, at 
which t ^ T. The value of the control at such times t G Ti\T would not influence the dynamics, or, maybe, only on 
[£*, +oo[x where t* = inf{s G T | s > £}. In this last case, note that £* G T and we can replace Ti by (Ti U {£*})\{£} 
without loss of generality. 
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2 . For every c S /T\{a}, q S AC([a, c]t, K") and (4) holds for A-a.e. t G [a, c)t- 

Existence and uniqueness of solutions (Cauchy-Lipschitz theorem on time scales) have been estab¬ 
lished in [15], and useful results are recalled in Section 4.1.2. 

Remark 2. The time scale Ti stands for the set of controlling times of the control system (4). 
If T = Ti, then the control is permanent. The case T = Ti = IR+ corresponds to the classical 
continuous case, whereas T = Ti = N coincides with the classical discrete case. If Ti £ T, the 
control is nonpermanent and sampled. In that case, the sampling times are given by t G RSi such 
that a{t) < (Ti{t) and the corresponding sampling time intervals are given by [t, ai{t))Y. The clas- 

sical optimal sampled-data control problem (OSDCP)p^y investigated in Theorem 1 corresponds 
to T = IR+ and Ti = NT, with T > 0. 

Remark 3. Let us consider two optimal control problems (OSDCP)!j[^ and (OSDCP)j^, posed 
on the same general time scale T for the trajectories, but with two different sets of controlling times 
Ti and T 2 , and let us assume that T 2 C Ti. We denote by $1 and $2 the corresponding mappings 
from T to Ti and T 2 respectively. If ui G L|?(Ti,Il) is an optimal control for (OSDCP)J^ and 
if there exists U 2 G L|?(T 2 ,n) such that ^2 (t) = Ui(t) for A-a.e. t G T, it is clear that U 2 is an 
optimal control for (OSDCP)^^. We refer to Section 3.1 for examples. 

Remark 4. The framework of (OSDCP)J^ encompasses optimal parameter problems. Indeed, 
let us consider the parametrized dynamical system 

A), A-a.e. tGT, (5) 

with A G n. Then, considering Ti = {a} U [b, -|-oo[t, (4) coincides with (5) where u{a) plays the 
role of A. In this situation, Theorem 3 (stated in Section 2.3) provides necessary conditions for 
optimal parameters A. We refer to Section 3.1 for examples. 

Remark 5. A possible extension is to study dynamical systems on time scales with several 
sampled-data controls but with different sets of controlling times: 

A-a.e. t G T, 

where Ti and T 2 are general time scales contained in T, and $1 and $2 are the corresponding 
mappings from T to Ti and T 2 . Our main result (Theorem 3) can be easily extended to this 
framework. Actually, this multiscale version will be useful in order to derive the transversality 
condition on the final time (see Remark 30). 

Remark 6. Another possible extension is to study dynamical systems on time scales with sampled- 
data control where the state q and the constraint function / are also sampled: 

q^it) = tGT, 

where Ti, T 2 and T 3 are general time scales contained in T, and $ 1 , $2 and $3 are the correspond¬ 
ing mappings from T to Ti, T 2 and T 3 respectively. In particular, the setting of [16] corresponds 
to the above framework with T = IR+ and Ti = T 2 = T 3 a general time scale. 

Although this is not the main objective of our paper, we provide hereafter a result stating 
the existence of optimal solutions for (OSDCP)!J^, under some appropriate compactness and 
convexity assumptions. Actually, if the existence of solutions is stated, the necessary conditions 
provided in Theorem 3, allowing to compute explicitly optimal sampled-data controls, may prove 
the uniqueness of the optimal solution. We refer to Section 3.1 for examples. 

Let A4 stand for the set of trajectories q, associated with 6 G T and with a sampled-data control 
u G L(]P(Ti, n), satisfying (4) A-a.e. on [a, 6 )t and g{q{a), q{b)) G S. We define the set of extended 
velocities W{t,q) = {{f{t,q,u),f°{t,q,u))^ | m G 11} for every {t,q) G T x M". 
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Theorem 2. If is compact, A4 is non-empty, Uglloo + ^ < M for every q G Ai and for some 
M > 0, and ifW{t,q) is convex for every {t,q) € T x M”, then (OSDCP)J^ has at least one 
optimal solution. 

The proof of Theorem 2 is done in Section 4.4. Note that, in this theorem, it suffices to assume 
that g is continuous. Besides, the assumption on the boundedness of trajectories can be weakened, 
by assuming, for instance, that the extended dynamics have a sublinear growth at infinity (see, 
e.g., [25]; many other easy and standard extensions are possible). 

2.3 Pontryagin maximum principle for (OSDCP)^^ 

2.3.1 Preliminaries on convexity and stable fl-dense directions 

The orthogonal of the closed convex set S at a point x G S is defined by 

Os[a^] = {x' G M-’ I Vcc" € S, {x' ,x" — x)'Ri < 0}. 

It is a closed convex cone containing 0. 

We denote by ds the distance function to S defined by ds{x) = inf^j/gs ||a; — x'IIrj, for every 
a; S . Recall that, for every x gW , there exists a unique element Ps(a;) G S (projection of x onto 

S) such that ds(a^) = ||a;—Ps(a^)||K3 ■ It is characterized by the property (a;—Ps(a;), a;'—P s(x))h3 < 0 

for every x' G S. In particular, x — Ps(a;) G C>s[Ps(3^)]- The function Ps is 1-Lipschitz continuous. 
We recall the following obvious lemmas. 

Lemma 1. Let (xk)keN be a sequence of points ofW and (Cfc)fc 6 N be a sequence of nonnegative 
real numbers such that Xk ^ x G S and Ck{xk — Ps(a^fe)) x' G as k ^ +oo. Then x' G OsN- 

Lemma 2. The function dg : a: i—>■ ds{x)‘^ is differentiable on with ddg(a:)(a:') = 2(x — 
Ps(a:),a:')BJ- 

Hereafter we recall the notion of stable H-dense directions and we state an obvious lemma. We 
refer to [16, Section 2.2] for more details. 

Definition 1. Let v G LI. A direction y G LI is said to be a stable H-dense direction from v if there 
exists e > 0 such that 0 is not isolated in {a G [0,1], v'-\-a{y — v') G H} for every v' G BRm(z;, ejnfl. 
The set of all stable H-dense directions from v is denoted by D^,^g(u). 

Lemma 3. If LI is convex, then D^g^g(z)) = LI for every v G LI. 

2.3.2 Main result 

Recall that g is said to be submersive at a point (< 71 ,( 72 ) G R" x K” if the differential of g at this 
point is surjective. We define the Hamiltonian id : T x R" x R" x R x R™ —R of (OSDCP)!j[^ 
by H{t,q,p,p°,u) = {p,f{t,q,u))R^ p° f{t,q,u). 

Theorem 3 (Pontryagin maximum principle for (OSDCP)^^). If a trajectory q*, defined on 
[a, 6 *]t and associated with a sampled-data control u* G L^JP(Ti,r2), is an optimal solution of 
(0SDCP)!J^^, then there exists a nontrivial couple {p,p^), wherep G AC([a, 5*]t, R") (called adjoint 
vector) and p^ < 0 , such that the following conditions hold: 

• Extremal equations: 

= - 7 ^{t,q*{t),p'^{t),p°,u*'^{t)), p^{t) = — —{t,q*{t),p'^{t),p°,u*'^{t)), (6) 


for A-a.e. tG [a, 6 *)t. 
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• Maximization condition: 


— For Ai-a.e. s S [a, &*)ti H RDi, we have 

u*{s) S argmaxiJ(s, g*(s),p(s),p°, z). 
zGO 


— For every r S [a, 6 *)ti H RSi such that cri(r) < b*, we have 


[r,(Ti(r))T 


dH 

du 


(,T,q*{T),p'^{T),p°,u*{r)) At , y 



( 7 ) 


for every y € In the case where r G [a, &*)ti H RSi with cri(r) > b*, the 

above maximization condition is still valid provided (Ti(r) is replaced with b*. 


• Transversality conditions on the adjoint vector: 

If g is submersive at {q*{a), q*{b*)), then the nontrivial couple {p,p^) can he selected to satisfy 

where -if G Os[g{q*{a),q*{b*))]. 

• Transversality condition on the final time: 

If the final time is left free in the optimal control problem (OSDCP)^^, if b* belongs to the 
interior o/T (for the topology ofM.), and if f and /° are of class with respect to t in a 
neighborhood of b*, then the nontrivial couple (PiP^) can be moreover selected such that the 
Hamiltonian function t i—> H{t,q*{t),p{t),p^,u*’^{t)) coincides almost everywhere, in some 
neighborhood of b*, with a continuous function vanishing at t = b*. 

In particular, if u*^{t) has a left-limit att = b* (denoted by u*‘^{b*_)), then the transversality 
condition can he written as 


H{b*,q*{b*),p{b*),P°,u*^{b*_)) = 0. 


Theorem 3 is proved in Section 4. Several remarks are in order. 

Remark 7. As is well known, the nontrivial couple (p,p°) of Theorem 3, which is a Lagrange 
multiplier, is defined up to a multiplicative scalar. Defining as usual an extremal as a quadruple 
{q,p,p^,u) solution of the extremal equations (6), an extremal is said to be normal whenever 
yf 0 and abnormal whenever = 0. In the normal case p^ ^ 0, it is usual to normalize the 
Lagrange multiplier so that = —1. 

Remark 8 . Theorem 3 encompasses the time scale version of the PMP derived in [16] when 
the control is permanent, that is, when Ti = T. Indeed, in that case, for every r G [a, &*)t H RS, 
r G [a, 5 *)ti nRSi and cri(r) = cr(r) < b*. Then the condition (7) can be written as the nonpositive 
gradient condition 

(^^{r,q*{r),p{cr{r)),p°,u*{r)) , y-u*{r)^ <0, 

for every y G D^j^|3('u*(r)). Moreover, in the case of a free final time, under the assumptions made 
in the fourth item of Theorem 3, b* also belongs to the interior of Ti = T, and then in that case 
we recover the classical condition 

maxH{b*,q*{b*),p{b*),p°,z) = 0. 
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A fortiori, Theorem 3 encompasses both the classical continuous-time and discrete-time versions 
of the PMP, that is respectively, when T = Ti = M+ and T = Ti = N. 

Remark 9. If the final time is free, under the assumptions made in the fourth item of Theorem 3, 
and if moreover b* ^ Ti (or if b* is a left-scattered point of Ti), is constant on [pi{b*),b*)j 
where pi{b*) = maxjs e Ti | s < b*} < b* (and thus, in particular, u*'^{t) has a left-limit at 
t = b*), and therefore H{b*,q*{b*),p{b*),p°,u*{pi{b*))) = 0. This is similar to the situation of 
Theorem 1. 

Remark 10. Let us describe some typical situations of terminal conditions g{q{a),q{b)) € S in 
(OSDCP)q[.^, and of the corresponding transversality conditions on the adjoint vector. 

• If the initial and final points are fixed in (OSDCP)^^ , that is, if we impose q{a) = qa 
and q{b) = qb, then j = 2 n , 5(91,92) = (91,92) and S = {9a} x {qb}- In that case, the 
transversality conditions on the adjoint vector give no additional information. 

• If the initial point is fixed, that is, if we impose q{a) = qa, and if the final point is left free 
in (OSDCP)t^, then j = n , 5(91,92) = 9i and S = { qa }- In that case, the transversality 
conditions on the adjoint vector imply that p{b*) = 0. Moreover, we have p^ 0^ and we 
can normalize the Lagrange multiplier so that p^ = —\ (see Remark 7). 

• If the initial point is fixed, that is, if we impose 9 ( 0 ) = 9 ^, and if the final point is subject to the 

constraint G{q{b)) G (IR+)*^ in (OSDCP)j^, where G = (G^,... ,G^) : K" —>■ is of class 

and is submersive at any point 92 G G“^((M+)^), then j = n + k, 5(91,92) = (9i,G(92)) 
and S = { 9 a} x (]R+)^. The transversality conditions on the adjoint vector imply that 

k 

p(5*) = ^A.a,,G*(9*(6*)), 

1=1 


with Ai > 0 , f = 1 ,..., fc. 

• If the periodic condition q[a) = q{b) is imposed in (OSDCP)^^, then j = n, 5(91,92) = 91—92 
and S = (Oj. In that case, the transversality conditions on the adjoint vector yield that 
p{a) =p{b*). 

We stress that, in all examples above, the function 5 is indeed a submersion. 

Remark 11. In the case where 5 is not submersive at {q*{a),q*{b*)), to obtain transversality 
conditions on the adjoint vector. Theorem 3 can be reformulated as follows: 

If a trajectory q*, defined on [a, &*]t and associated with a sampled-data control u* G 
L“(Ti,n), is an optimal solution o/(OSDCP)^^, then there exists a nontrivial couple 
(V'jP®) G X M, with —ip G Os[g{q*{a),q*{b*))] and p^ < 0, and there exists p G 
AC([a, &*]t, K") such that the extremal equations, the maximization conditions and the 
transversality conditions are satisfied. 

However, with this formulation, the couple {p,p^) may be trivial^ and, as a consequence, the result 
may not provide any information. We refer to Sections 4.3.3 and 4.3.4 for more details. 

^Indeed, if p® = 0, then the adjoint vector p is trivial from the extremal equation and from the final condition 
p{b*) = 0. This leads to a contradiction since the couple (p,p^) has to be nontrivial. 

"^Indeed, if = 0 and j) belongs to the kernel of {dq 2 g{q* (a,), q* (b*)))^, then the couple {p,p^) is trivial. This 
situation leads to a contradiction if g is submersive at (q* (a), q* (b*)). 
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Remark 12. In this paper, the closedness of is used in a crucial way in our proof of the PMP. 
Indeed, the closure of allows us to define the Ekeland functional on a complete metric space (see 
Section 4.3.2). However, if the initial point is fixed, that is, if we impose q{a) = Qa, and if the 
final point is left free in (OSDCP)^^ , then Theorem 3 can be proved with a simple calculus of 
variations, without using the Ekeland variational principle. In this particular case, the closedness 
assumption can be removed. 

Remark 13. If the cost functional to be minimized in (OSDCP)j^ . i® f[a,b)r + 

£(b, q(a), q(b)), where t': T x M” x K" —)■ K is continuous, of class in (gi, 92 ), then the transver- 
sality conditions on the adjoint vector become 

p(a) = - (dg^g(q*(a),q*(b*)))^ ip - p°dgj(b*,q*(a),q*(b*)), 

and p(b*) = (dg^g(q*(a), q*(b*)))^ ip + p°dgj(b*, q*(a), q*(b*)). 

Moreover, in the fourth item of Theorem 3, if £ is of class in a neighborhood of b*, the transver- 
sality condition on the final time must be replaced by: 

The nontrivial couple {p,p^) can be selected such that the Hamiltonian function t !—>■ 
H{t,q*{t),p{t),p^,u*‘^{t)) coincides almost everywhere, in some neighborhood of b*, 
with a continuous function that is equal to —p^db£{b*, q*{a), q* {b*)) at t = b*. 

To prove this claim, it suffices to modify accordingly the Ekeland functional in the proof of Theo¬ 
rem 3 (see Section 4.3.2). 

3 Applications and further comments 

In this section, we first give, in Section 3.1, a very simple example of an optimal control problem 
on time scales with sampled-data control, that we treat in details and on which all computations 
are explicit. The interest is that this example provides as well a simple situation where it is evident 
that some of the properties that are valid in the classical continous-time PMP do not hold anymore 
in the time-scale context. We gather these remarks in Section 3.2. 

3.1 A model for optimal consumption with sampled-data control 

Throughout this subsection, T and Ti are two time scales, unbounded above, satisfying Ti C T, 
minT = minTi = 0 and 12 S T. In the sequel, we study the following one-dimensional dynamical 
system with sampled-data control on time scales: 

q^{t) = u‘^{t)q{t), A-a.e. < e [0, I2 )t, (9) 

with the initial condition q{0) = 1, and subject to the constraint u{t) G [0,1] for Ai-a.e. t G 
[0, I2)ti. Since the final time 6 = 12 is fixed, we can assume that 12 G Ti without loss of 
generality. 

The above model is a classical model for the evolution of a controlled output of a factory during 
the time interval [0, I2]t (corresponding to the twelve months of a year). Precisely, q(t) G M stands 
for the output at time t G T and u(t) G [0, 1] stands for the fraction of the output reinvested 
at each controlling time t G Ti. We assume that this fraction is sampled at each sampling time 
t e Ti such that t G RSi and a(t) < cri(t), on the corresponding sampling interval [t, tTi(t))T. In 
the sequel, our goal is to maximize the total consumption 

C{u)=f {1 - {T))q{T) At. (10) 

i[0.12)T 
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In other words, our aim is to maximize the quantity of the output that we do not reinvest. 

Remark 14. In the continuous case and with a permanent control (that is, with T = Ti = K"''), 
the above optimal control problem is a very well-known application of the classical Pontryagin 
maximum principle. We refer for example to [48, Exercice 2.3.3. p.82] or [40, p.92]. In this section, 
our aim is to solve this optimal control problem in cases where the control is nonpermanent and 
sampled. We first treat some examples in the continuous-time setting T = K+ and then in the 
discrete-time setting T = N. 

The above optimal sampled-data control problem corresponds to (OSDCP)J^ with n = m = 
j = 1, a = 0, b — 12 (fixed final time), = [0,1] (convex), g{qi,q 2 ) = 9 i and S = {1} (fixed initial 
value q{0) = 1 and free final value 9 ( 12 )), f{t,q,u) = uq and f^{t,q,u) = {u — l)q (our aim is to 
minimize —C(u)). 

Since / and /° are affine in u and since ft is compact. Theorem 2 asserts that (OSDCP)!Ji^ 
admits an optimal solution q*, defined on [ 0 , 12]t and associated with a sampled-data control 
u* G L([P(Ti, [0,1[). We now apply Theorem 3 in order to compute explicitly the values of u* at 
each controlling time t G [0, 12)ti. The nontrivial couple (p,p°) satisfies = —1 and p(12) = 0 
(see Remark 10). The adjoint vector p G AC([0, 12]t,K) is a solution of 

p^{t) = -u*‘^{t)p^t) + - 1, A-a.e. t G [0, 12)t. (11) 

Moreover, one has the following maximization conditions: 

1. for Ai-a.e. s G [0, 12)ti n RDi, 

u*{s) G argmax(z(p(s) — 1) -I- l)q*{s). (12) 

2:e[0.1] 

2. for every r G [0, 12)ti n RSi, 

f 9 *('r)(p'^(T) - l)(y - ii*(r)) At < 0, (13) 

for every y G D^ab('“*(^)) = = [0,1] (since ft is convex, see Lemma 3). 

Since q* is a solution of (9) and satisfies q*{0) = 1, one can easily see that q* is monotonically 
increasing on [0, 12]t and then q* has positive values. From (11) and since p{12) = 0, one can 
easily obtain that p is monotonically decreasing on [ 0 , 12 ]x and then p has nonnegative values. 


3.1.1 Continuous-time setting T = M+ 

Case Ti = R+ (permanent control). Since q* has positive values, one can conclude from (11) 
and (12) that: 


u*{t) 


1 

0 


for a.e. t such that p{t) — 1 > 0, 
for a.e. t such that p{t) — 1 < 0, 


pit) 


—pit) for a.e. t such that u*{t) = 1, 
— 1 for a.e. t such that u*{t) = 0. 


Since _p(12) = 0, one can easily prove that the optimal (permanent) control u is unique and given 
by u*(t) = 1 for a.e. t G [0,11) and u*{t) = 0 for a.e. t G [11,12). The associated optimal 
consumption is C{u*) = ~ 59874.142. We refer to [48, Exercice 2.3.3. p.82] or [40, p.92] for 

more details. 


13 


Case Ti discrete (sampled-data control). Solving the differential equations (9) and (11) 
leads to q*it) = 9 *(cri(r))e“*(’')(‘-'^iW) and 


P'^{t)=p{t) 


p(cri(r))e-“*W(‘-‘"iW) + (1 - e-“*W(‘-'^iW)) if M*(r) ^ 0 , 

p{ai{r)) + ai{r) — t ifM*(r) = 0 , 


for every r G [0, 12)ti and every t G [r,cri(r)]. Then, (13) can be written as q*{cri{r)){y — 
u*{r))Tr{u*{r)) < 0 , where T^ : [ 0 , 1 ] —^ M is the continuous function given by: 


rr(a:) 


_ g _p(cri(r))) - 1)^ 

pi{r) (^p{ai{r)) + - 1 ^ 


if a: 7 ^ 0 , 


if X = 0 . 


Since (13) holds true for every y G [0,1] and since q* has positive values, the following properties 
are satisfied for every r G [0, 12)ti : 

• if Tj. has negative values on ( 0 , 1 ], then u*{r) = 0 ; 

• if Tj. has positive values on [ 0 , 1 ), then u*{r) = 1 ; 

• if rr(0) > 0 and rj.(l) < 0, then u*{r) G (0,1) is a solution of the nonlinear equation 
rj.(x) = 0. 

Note that T,, depends only on pii{r) and p(cri(r)). As a consequence, the knowledge of the value 
p( 12 ) = 0 and the above properties allow to compute u*{ro) where xq is the element of [ 0 , 12 )ti 
such that cri(ro) = 12 (and pi{ro) = 12 — ro). Then, the knowledge of u*(ro) allows to compute 
p(ro) from (11). Then, the knowledge of p(ro) and the above properties allow to compute u*{ri) 
where ri is the element of [0, 12 )ti such that o'i(ri) = tq (and pi{ri) = tq — ri), etc. Actually, 
this recursive procedure allows to compute u*{r) for every r G [0, 12)ti. Numerically, we obtain 
the following results: 


II 

u*{t) = 1 

1 if t G {0,1,..., 10} 

0 ift=ll 

C{u*) = ~ 59874.142 

CSI 

II 

u*{t) = j 

1 if t G 10,2,4,6,8} 

0 if t = 10 

C{u*) = 2ei° ~ 44052.932 

Ti = 3N 

II 

* 

1 if tG {0,3, 6 } 

0.4536 if t = 9 

C{u*) ~ 28299.767 

II 

u*{t) = 

J 1 iftG{0,4} 

} 0.6392 iff = 8 

C{u*) ~ 20013.885 

II 

u*(t) = 

J 1 if t = 0 

■ } 0.4536 if t = 9 

C{u*) ~ 28299.767 

Ti = 12N 

u*(0) ~ 0.9083 

C{u*) ~ 5467.24 

Ti = 12NU{10,11.5} 

u*{t) = 

[1 if t = 0 

\ 0.9072 if t = 10 

[ 0 if t = 11.5 

C{u*) ~ 49476.604 


Remark 15. In this example. Theorem 2 states the existence of an optimal solution. In all cases 
above studied, the Pontryagin maximum principle proves that the optimal solution is moreover 
unique. 

Remark 16. The case Ti = N can be easily deduced from the permanent case Ti = K+ (see 
Remark 3). Similarly, the case Ti = 9N can be deduced from the case Ti = 3N. 
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Remark 17. The case Ti = 12N corresponds to an optimal parameter problem (see Remark 4). 

Remark 18. For the needs of Section 3.2, let us give some details on the case Ti = 2N. In 
that case, q*{t) = e* on [0,10] and q*(t) = on [10,12]. Moreover, p{t) = 2e^°“* on [0,10] and 
p{t) = 12 — t on [10,12]. Hence, for every t G [10,11), argmax^gf; i7(t, q* (t),p(t),p°, 2 ) = {1}. 
The Hamiltonian t !->■ H{t, g*(t),p(t),p°, is equal to t !->■ 2e^° a.e. on [0,10) and is equal to 

1 1— >■ a.e. on [10,12). Finally, the maximized Hamiltonian 1 1— >■ max^go H{t, q*(t) , p'^(t),p^, z) is 
equal to t !->■ 2 e^° on [ 0 , 10 ), to t i-G (12 — f)e^° on [ 10 , 11 ) and to t !->■ on [ 11 , 12 ). 

Case Ti hybrid (sampled-data control). In this paragraph, we study the hybrid case Ti = 
[0,6] U {10} U [11.5, +00). Similarly to the permanent case Ti = IR+, one can easily conclude 
from ( 12 ) that: 


. _ r 1 for a.e. t G [0,6) U [11.5,12) such that p{t) — 1 > 0, 

^ {0 for a.e. t G [0,6) U [11.5,12) such that pit) — 1 < 0. 

Since p(12) = 0, one can easily prove that u*{t) = 0 for a.e. t G [11.5,12), and then p(11.5) = 0.5. 
From the knowledge of p(11.5) = 0.5 and using similar arguments than in the previous paragraph, 
one can compute m*( 10) ~ 0.9072. From (11), it gives p(10) ~ 2.2462. From the knowledge 
of p(10) ~ 2.2462, one can compute u*( 6 ) = 1. From (11), it gives p( 6 ) ~ 122.6402. Since 
p is monotonically decreasing, we conclude that p{t) — 1 > 0 for every t G [0,6). We finally 
conclude that the optimal sampled-data control u* is unique and is given by u*{t) = 1 for a.e. 

t G [0,6), u*( 6 ) = 1, u*(10) ~ 0.9072 and u*{t) = 0 for a.e. t G [11.5,12). The associated optimal 

consumption is C{u*) ~ 49476.604. 

Remark 19. The discrete case Ti = 12N U {10,11.5} can now be seen as a consequence of the 
hybrid case Ti = [0,6] U {10} U [11.5, -foo) (see Remark 3). 


3.1.2 Discrete-time setting T = N 

In the discrete-time setting T = N, similarly to the continuous-time one, we can prove that (13) 
can be written as q*{r){y — u*{r))Ar{u*{r)) < 0 , where : [ 0 , 1 ] —>■ K is the continuous function 
given by: 


Ar{x) 


1 - ( 1 -I- 1 (^ 1 -|-/ri(r)a:(x(l-p(cri(r))) - 1 )-l-x^ 
pi{r) ^p(cri(r)) -h 


if X ^ 0 , 


if X = 0 . 


The same recursive procedure (than in the previous section) allows to compute u*(r) for every 
r G [0, 12)ti . Numerically, we obtain the following results: 


Ti = 2N 

u*[t) = <j 

1 if t G {0,2,4, 6,8} 

0 if t = 10 

C{u*) = 2^1 = 2048 

II 

CO 

u*{t) = 

1 

1 if fG {0,3,6} 

0 if f = 9 

C{u*) = 3 • 29 = 1536 

II 

u*{t) = 

} 1 if t G {0,4} 

( 0.2886 iff = 8 

C{u*) ~ 1108.882 

Ti = 6 N 

u*it) = 

= < 

[ 1 if t = 0 

[ 0.5725 iff = 6 

Ciu*) ~ 674.787 

Ti = 12N 


u 

*(0) ~ 0.8145 

C'(u*) ~ 159.647 
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Remark 20. In the permanent discrete case T = Ti = N, the optimal control is not unique. 
Indeed, every control u* satisfying u*{t) = I for every t S {0,1,..., 9}, u*(10) G [0,1] and u*(ll) = 
0 is optimal. Let us give some details on the proof and let us note that the PMP does not provide 
any constraint on the value rt*(10) in that case. 

With r = 11 and p(12) = 0, has negative values on (0,1] (constantly equal to —1), then 
rt*(ll) = 0 from (13). From (11), we compute p(ll) = 1. With r = 10 and p(ll) = 1, A,, is 
constantly equal to 0 and then (13) does not provide any constraint on the value m*( 10). Actually, 
it does not matter since p(10) can still be computed from (11) (p(10) = 2) and the recursive 
procedure can be pursued. We obtain u*{t) = 1 for every t G {0,1,..., 9}. From (10), we have 

C{u*) = (1 - u*(10))g*(10) + q*{n) = (1 - u*(10)) • 2^° + (1 + u*(10))g*(10) = 2 • 2^° = 

Finally, if u*(ll) = 0 and u*(t) = 1 for every t G {0,1,..., 9}, the value of m*( 10) G [0,1] does not 
influence C{u*). This concludes the proof and the remark. 

Remark 21. The case Ti = 2N can be seen as a consequence of the permanent case Ti = N (see 
Remarks 3 and 20). 

Remark 22. The case Ti = 12N corresponds to an optimal parameter problem (see Remark 4). 

3.2 Non-extension of several classical properties 

In this section, we recall some basic properties that occur in classical optimal control theory in 
the continuous-time setting and with a permament control, that is, with T = Ti = M’*'. Our 
aim is to discuss their extension (or their failure) to the general time scale setting and to the 
nonpermament control case. We will provide several counterexamples in the discrete-time setting 
with a permanent control (T = Ti = N) and in the continuous-time setting with a nonpermanent 
control (Ti 5 T = IR+). In the following paragraphs, except the last one, the hnal time & G T can 
be fixed or not. 

Point-wise maximization condition of the Hamiltonian. In the case T = Ti = K+, 

it is well known that an optimal (permanent) control u* satisfies the maximization condition 
u*{t) G argmax^gQ iL(t, ( 7 *(t),p(<),p°, z) for a.e. t G [0,5*). We refer to [16, Example 7] for a 
counterexample showing the failure of this maximization condition in the case T = Ti = N. We 
refer to Remark 18 for a counterexample in the case Ti ^ T = R+. 

Continuity of the Hamiltonian. In the case T = Ti = R+, it is well known that the Hamilto¬ 
nian function t H{t,q*{t),p{t),p^,u*{t)) coincides almost everywhere on [0,5*] with the contin¬ 
uous function t i— uia:x.zen H(t,q* (t),p{t),p^, z). Remark 18 provides a counterexample showing 
the failure of this regularity property in the case Ti ^ T = IR+. 

Remark 23. Nevertheless, in the case of a free final time, under the assumptions of the fourth 
item of Theorem 3, the Hamiltonian function t i— H{t,q*{t),p{t),p'^,u*’^{t)) coincides almost 
everywhere, in some neighborhood of 5*, with a continuous function. 

The autonomous case. In the case T = Ti = IR+, if the Hamiltonian H is autonomous (that 
is, does not depend on t), it is well known that the function t i-G- H{q* {t),p{t), p°, u*{t)) is almost 
everywhere constant on [0, 5*], this constant being equal to the maximized Hamiltonian. We refer 
to [16, Example 8] for a counterexample showing the failure of this constantness property in the 
case T = Ti = N, and we refer to Remark 18 for a counterexample in the case Ti ^ T = R+ (and 
there, even the maximized autonomous Hamiltonian is not constant). 
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Saturated constraint set fl for Hamiltonian affine in u. In this paragraph, we assume that 
n is convex. In the case T = Ti = K’*', if the Hamiltonian is affine in u, that is, if it can be written 
as 

H{t,q,p,p°,u) = {Hi{t,q,p,p°),u)Rrr. + H 2 {t,q,p,p°), 

one can easily prove that Hi{t,q*{t),p{t),p^) G On[u*{t)] for almost every t G [0,5*). It follows 
that an optimal (permament) control u* must take its values at the boundary of Q (saturation of 
the constraints) for almost every t G [0,5*) such that Hi{t, q*(t),p{t),p^) ^ 0®™. 

Remark 24. This classical property can be extended to the case T = Ti = N. Indeed, in that 
case, the nonpositive gradient condition is given by 

(^^{t,q*{t),p{t + l),p°,u*{t)),y-u*{t)^ = {Hi{t,q*{t),p{t+l),p°),y-u*{t))M^ < 0, 

for every j/ G H, that is, Hi{t,q*{t),p{t + l),p°) G On[u*{t)]. 

Remark 25. Remark 20 provides an interesting example in the case T = Ti = N. Indeed, in that 
case, the control defined by u*{t) = 0 for every t G {0,..., 9}, m*( 10) = 1/2 and m*( 11) = 1 is an 
optimal (permanent) control. However, it does not saturate the constraint set H at t = 10. It is 
not a surprise since, in that case, Hi{t, q*{t),p{t + 1 ),P°) = 0 at t = 10 . 

Note that, in the case Ti = 4N, Section 3.1.2 provides a counterexample showing the failure 
of this classical property in the case Ti ^ T = N. Similarly, Section 3.1.1 in the case Ti = 3N 
provides a counterexample in the case Ti ^ T = K+. 

Remark 26. Figure 1 represents the values of the optimal sampled-data control u* of Section 3.1.1, 
in the case Ti = 12N U {A}, where A is a parameter evolving in (0,12). In that case, u*{0) 
(resp., m*(A)) saturates the constraint set = [0,1] approximately for A G (0,11.9245) (resp., for 
A G (9.9866,12)). 



Figure 1: Optimal sampled-data control u*. 


Vanishing of the maximized Hamiltonian at the final time. In this paragraph, we assume 
that the final time is left free. In the case T = Ti = K+, under the assumptions of the fourth item 
of Theorem 3, it is well known that the maximized Hamiltonian vanishes at < = 5* (see Remark 8 ). 
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In Theorem 3, we have established that this property is still valid in the time scale setting under 
some appropriate conditions, the main one being that b* must belong to the interior of T. In the 
discrete case T = N, the interior of T is empty and then the latter assumption is never satisfied. 
The Hamiltonian at the final time may then not vanish, and we refer to [16, Example 8] for a 
counterexample with T = Ti = N. 


4 Proofs 

The section is structured as follows. Subsections 4.1, 4.2 and 4.3 are devoted to the proof of 
Theorem 3. In Subsection 4.1, we recall some known Cauchy-Lipschitz results on time scales and 
we establish some preliminary results on the relations between u and . In Subsection 4.2, we 
introduce appropriate needle-like variations of the control. Finally, in Subsection 4.3, we apply the 
Ekeland variational principle to an adequate functional in an appropriate complete metric space, 
and then we prove the PMP. In Subsection 4.4 (that the reader can read independently of the rest 
of Section 4), we detail the proof of Theorem 2. 

4.1 Preliminaries 

4.1.1 Relations between u and 

We start with a lemma whose arguments of proof will be used several times. 

Lemma 4. Let m G N* and let c < d he two elements o/T with c G Ti. Let u, v : [c,d)j^ —>■ M™ 
be two functions. Then, u = v Ai-a.e. on [c, (i)Ti if and only if A-a.e. on [c,d)j. 

Proof. Without loss of generality, we can assume that v is constant equal to Offm. Let us define 
A = {t G [c, d)Ti \ u{t) ^ ORm} and B = {t G [c, d)T \ u'^(t) Or™}. Since $(t) = t for every 

t G Ti, the inclusion A C B holds. Firstly, let us assume that /rAi(4) = 0. Hence, A C RDi 
and fj,Ai{A) = fJ.L{A) = 0. Since A c RDi c RD, we deduce that ^a(4) = yLL{A) = 0. On the 
other hand, for every t G B, $(t) G A C RDi, then t G Ti and t = 4>(t) G A. We conclude that 
A = B and fiA{B) = 0. Secondly, let us assume that ^a(R) = 0 and ^Ai(4) > 0. Since A C B, 
we deduce that ^a(4) = 0, H c RD and fJ.A{A) = ^l(4) = 0. Since yiL{A) = 0 and ^Ai(4) > 0, 
we conclude that there exists to G RSi n H C RSi n RD. Consequently, u* is constant (different 
of Or™) on [to,min(CTi(to))d))T- As a consequence, ytAiB) > min(CTi(to)) d) — to > 0. This raises a 
contradiction. □ 

Proposition 1. Let m G N* and let c < d be two elements o/T with c G Ti. 

1. For every u G L.][.^ ([c, d)Ti, R™), we have G L.][.([c, d)T, R”^) and 


lk^llLi([c,d)T.R™) ^ lkllLi^([c.d)Ti.R'")' (1^) 

2. For every u G L([P ([c, d)Ti, R™), we have G L([P([c, (i)T,R'") and 

ll'w‘^l|L“([c.d)T.R’") = |l'w||L“([c,d)Ti.R’-)- 

Proof. Let u G L.[j.^ ([c, d)Ti, R™). We first treat the /r a- measurability of u’^. From Lemma 4, we 
can consider that u is defined everywhere on [c,(i)Ti and is /iAi-measurable on Let us 

prove that is ^A-measurable on [c, d)^. We introduce d' = inf{s G Ti | s > d} G Ti. Note that 
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d' > d, with equality if and only if d G Ti. From [23], u is /xaj- measurable on [c, (i)Ti if and only 
if the extension u defined on [c, d') by 


u{t) = I 


u{t) if t £ Ti, 

u{r) if f G (r, CTi(r)), wherer G RSi, 


is /ii-measurable on [c,d'). By hypothesis, u is /iz,-measurable on [c, d') and consequently, the 
restriction is /ri-measurable on [c, d). Still from [23], is /iA-nieasurable on [c, d)T if and 

only if the extension defined on [c, d) by 



u^it) if t G T, 

u^{r) if t G (r,(T(r)), wherer G RS, 


is /Xi-measurable on [c, d). Hence, it is sufficient to see that . This is true since, for any 

t G [c, d), we have: 

• either t G Ti, and then t G T and thus = u{^{t)) = u{t) = u{t). 

• either t G T\Ti, and then t G (<i>(t), cri(d>(t))) where d>(t) G RSi and thus = u'^{t) = 

u($(t)) = u(t). 

• either t ^ T, and then t G (r, cr(r)) where r G RS and then t G (<i>(r), (Ti(<i>(r))) where 
$(r) G RSi and thus = u^{r) = w(<i)(r)) = u{t). 

This establishes the /XA-measurability of u’^ on [c, d)T- 

Considering ||u||r>ti instead of u, we can consider that m = 1 and u G L]r^([c,d)Ti,R''’)- 
From [23], we have 



Noting that (d' — d)?x(<i>(d)) > 0 concludes the proof of the first point. 

Let us prove the second point. Let u G L^ ([c, d)Ti, K™). The /x a- measurability of xx'^ is already 
proved since L|P ([c, d)Ti , K”^) C L.^^ ([c, d)Ti , R™)- Let M > 0 be a constant. With the same 
arguments as in the proof of Lemma 4, we can prove that ||xi(t)||Rm < M for Ai-a.e. t G [c, d)j^ if 
and only if ||?x*(t)||R™ < M for A-a.e. t G [c, d)T. As a consequence, we get xx^ G L“([c, d)T, R™) 
and ||xx‘^||l“(M)t,R-) = I|u||l“(M)ti,R-)- □ 

Remark 27. From the proof above, we see that the inequality (14) is an equality if and only if 
d G Ti or xx($(d)) = 0. Then, considering T = N, Ti = 2N, m = l, c = 0, d=l and xx the constant 
function equal to 1 provides a counterexample. Indeed, in that case, we have xx G L^^ ([c, d)Ti, R™) 



4.1.2 Recalls on A-Cauchy-Lipschitz results 

According to [15, Theorem 1], for every control xx G L|P (Ti, R'") and every initial condition qa G R", 
there exists a unique maximal solution of (4) such that q{a) = ga, denoted by q{-,u,qa), and 
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defined on a maximal interval, denoted by /t(m, ga)- The word maximal means that q{-,u,qa) is 
an extension of any other solution. Moreover, we recall that (see [15, Lemma 1]) 



Finally, either /t(u, ga) = T, that is, g(-,u, ga) is a global solution of (4), or If(u,qa) = [a, c)t 
where c is a left-dense point of T, and in that case, q{-,u,qa) is unbounded on /t(u, ga) (see [15, 
Theorem 2]). 

Definition 2. For a given 6 G T, a couple {u,qa) G L“(Ti,K'") x M" is said to be admissible on 
[a, 6]t whenever b G It{u, qa)- 

For a given 5 G T, we denote by UQ!’ the set of all admissible couples {u,qa) on [o, 6 ]t. It is 
endowed with the norm ||(M,ga)Lc'’ = IKIIli^([«.%,.R-) + Ikalk-- 

4.2 Needle-like variations of the control, and variation of the initial 
condition 

Throughout this section, we consider 6 G T and (m, ga) G UQ^. We are going, in particular, to 
define appropriate needle-like variations. As in [16], we have to distinguish between right-dense 
and right-scattered times, along the time scale Ti. We will also define appropriate variation of the 
initial condition. 

In the sequel, the notation || • || stands for the usual induced norm of matrices in K"’", M"’™ 
and 

4.2.1 General variation of {u,qa) 

In the first lemma below, we prove that lAQ^ is open. Actually we prove a stronger result, by 
showing that U contains a neighborhood of any of its point in L][. x M" topology, which will be 
useful in order to define needle-like variations. 

Lemma 5. Let R > ||M||L“([a,b)Ti,R'")- T/iere exist vr > 0 and rjn > 0 such that the set 



is contained inUQ’’. 


Before proving this lemma, let us recall a time scale version of Gronwall’s Lemma (see [10, 
Chapter 6.1]). The generalized exponential function is defined by 6^(1, c) = exp(/j^ At), 

for every L > 0, every c G T and every t G [c, -|-oo)t, where = \og{\+L/^{ t) whenever 

/r(r) > 0, and = L whenever ^{t) = 0 (see [10, Chapter 2.2]). Note that, for every L >0 

and every c G T, the function eL{-,c) is positive and increasing on [c, -|-oo)t- 

Lemma 6 ([10]). Let c < d be two elements o/T, let Li and L 2 be two nonnegative real numbers, 
and let q G C([c, d]T,R) satisfying 0 < q{t) < Li + L 2 ffc t)t At, for every t G [c, (i]T. Then 
0 < q{t) < LieL 2 {t,c), for every t G [c,d]T- 

Proof of Lemma 5. By continuity of q{-, u, qa) on [a, 6]t, the set 


Kr = {{t,x,v) G [a,5]T X M” X BR™(0,i?) ] JJa; - q{t,u,qa)\\wr, < 1} 
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is a compact subset of T x M" x K"*. Therefore \\df /dq\\ and ||9//9u|| are bounded on Kji by 
some Lji > 0 and then, from convexity and from the mean value inequality, it holds: 


\\f{t,X2,V2) - /(t,a:i,ui)||K- < T_R(||a ;2 - a;i||B’* + ||i ’2 - t'llk-), (15) 

for all (t, xi, ui), (t, X 2 ,V 2 ) G K^. Let Vfi> 0 and 0 < 77 ^ < 1 such that {rjn + VRLji)eLn{h, a) < 1. 

Let (u', q'g) G g^). Our aim is to prove that b G q'^). By contradiction, assume that 

the set 

A = {tG lT{u,q^) n [a, 6 ]T | \\q{t,u,q^) - g(t, u, ga)liH" > 1} 

is not empty and let to = inf A. Since T is closed, to G q'^) H [a, 6 ]t and [a, to]T C It{u', q'^) O 
[a, 6 ]t- If to is a minimum then ||g(to, u', g),) — q(to,u,qa)\\s.^ > 1- If to is not a minimum then 
to G RD and by continuity we have |ig(to,ga) — q{to,u,qa)\\Ts.^ > I- Moreover one has to > a 
since \\q{a,u',q'^) - g(a, u, ga)||B" = ha - dalk" < < I- Hence ||g(r, u', g),) - g(T,u,ga)|k" < 1 

for every t G [a,to)T- Therefore {T,q{T,u\q'h,u'’^{T)) and (r, g(T, m, g^), u^(t)) are elements of 
Kr for A-a.e. r G [o,to)T- Since one has 

u', q'a) - u, qa) =qa-qa+ ( {/{t, q{T, u', q'h, u'‘^{r)) - /(t, g(T, u, qa),u‘^{T))) At, 

J [a,i)T 

for every t G Iy{u', q'h O [a, 6]t, it follows from (15) that, for every t G [a, to]T, 


- qit,u,qa)\\m.^ < ha - 9alk" +Lr \\u''^{t) - u‘^(T)|k™ At 

+ Lr ||g(r,M',g(j) - g(r,u,ga)|k" At, 

d[a,t)T 

which implies from Lemma 6 that, for every t G [a,to]T, 

lk(^. q'a) - q{t, u, ga)|k" < iha - 9a|k" + LrWu"^ - ||Li([a.b)T,B-))eLH (&, o), 

which finally implies from Proposition 1 that, for every t G [a,to]T, 

h(.'t,u',q'h - g(t,M,ga)|k" < i\Wa “ 9a|k" + Lr\\u' - u||Li^([a,f,)Tj,B™))eL„(5,a) 

< hR + URLR)eLhb,a) < 1 . 

This raises a contradiction at t = to- Therefore A is empty and thus g(-,u',g(,) is bounded on 
Ij{u',q'h n [a, 5 ]t. It follows from [15, Theorem 2] that b G If{u',q'h, that is, {u',q'h G UQ!’. □ 

Remark 28. Let R > Iklk” ([a,b)Ti .B”*) and {u',q'h G ER{u,qa)- With the notations of the above 
proof, since [a, 6 ]t C If{u',q'h and A is empty, we infer that ||g(t, u', g),) — g(t,u, gQ)|k" < Ij for 
every t G [a, 6 ]t. Therefore (r, g(T, u', q'h, G Kr for every («', q'h G Eij(M, g^) and for A-a.e. 

T e [a,6)T. 

Lemma 7. Let R > Iklk^ .S’") ■ The mapping 

FR{u,qa)-. (E7j(u,ga), II • IkcO —(C([a,5]T,K”), II • Iloo) 

W^q'a) '—^ 9 (-,u',g;) 

is Lipsehitzian. In particular, for every {u',q'h G RiR{u,qa), q{-,u',q'h converges uniformly to 
q{-,u,qa) on [a, 6 ]t when u' tends to u in L^kk, 5 )ti , R"*) and cf^ tends to qa in K". 
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Proof. Let {u',q'^) and (u”,q”) be two elements of PiR{u,qa) C UQ^. It follows from Remark 28 
that {T,q{T,u'',q”),u'''^{T)) and (r, q(T, rt', q^), M'^(r)) are elements of for A-a.e. r € [a,b)j. 
Following the same arguments as in the previous proof, it follows from (15), from Lemma 6 and 
from Proposition 1 that, for every t G [a,5]T, 


h{t,u",q'f) - qit, u\q'^] 


< (Ik" 


+ Lr\\u" - m'IIli ([a,f,)Ti,R™))eiH(k a). 


The lemma follows. 


□ 


4.2.2 Needle-like variation of m at a point r G RSi 

Let r G [a, 6 )ti H RSi and let y G M™. We define the needle-like variation 11 = (r, y) of u at r by 


un{t,a) 


u(r) + a{y — u(r)) if t = r, 
ult) if t ^ r. 


for Ai-a.e. t G [a, 6 )ti and for every a G [0,1]. In the sequel, let us denote by cr)'(r) = min(cri(r), b). 

Lemma 8. There exists 0 < oq < 1 such that (Mn(-,o),qa) G UQ’’ for every a G [0,Q!o]- 

Proof Let R = max(||M||L~([a_b).,^_Rr„), ||u(r)||R™-h||y||R^)-bl > ||■u||L“([a.h)T^,R™)• We use the nota¬ 
tions RTr, Lr, vr and rjR, defined in Lemma 5 and in its proof. One has ||un(', Q!)||L“([a,&)Tj ,R'") A R 
for every a G [0,1], and 


||■u^(•,a) - = /ri(r)||Mn(7’,<a) - ^(OIIr™ = afj.i{r)\\y - u{r)\\M^. 

Hence, there exists 0 < oq A 1 such that ||Mn(-,a) — m||li ([a,h)T^.R"*) — every a G [0,Q!o]) 

and hence (itn(’, a), qa) G Er(m, q^). The claim follows then from Lemma 5. □ 

Lemma 9. The mapping 

Fniu,qa)-- ([0,ao],|-|) —^ (C([a, 6]t,M"), || • ||oo) 

a I—S> q{-,un{-,a),qa) 


is Lipschitzian. In particular, for every a G [0,ao]> 9(’) wn(-, o), qa) converges uniformly to 
q{-,u,qa) on [a, 6 ]t as a tends to 0. 

Proof. We use the notations of the proof of Lemma 8. It follows from Lemma 7 that there exists 
C > 0 (the Lipschitz constant of FR{u,qa)) such that 

\\q{-,un{-,0!2),qa) - < 7 (',Mn(-,ai),ga)||oo < C'||(un(-,a2),ga) - {uni-,ai),qa)\\uQ>’ 

= C\a 2 - ai\yi{r)\\y - u(r)||R™, 


for all ai and 02 in [0, ao]. The lemma follows. □ 

We define the so-called first variation vector /in(-, u, qa) associated with the needle-like variation 
n = (r, y) as the unique solution on [r, (Tj(r)]x of the linear A-Cauchy problem 

df df 

h^{t) = -^it,qit,u,qa),u{r)) h{t) + -^{t,q{t,u,qa),u{r)) {y - u{r)), h{r) =0. (16) 

The existence and uniqueness of hii{-,u,qa) are ensured by [15, Theorem 3]. 
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Proposition 2. The mapping 


Fn{u,qa) : 


([0,ao],|-|) 

a 



(C([r,ar(r)]T,M"),||-||oo) 

q{-,uu{-,a),qa) 


is differentiable at 0, and one has DFuiu, qa){0) = hii[-,u,qa)- 

Proof. We use the notations of the proof of Lemma 8. Recall that (r, q{T, uu{-, Oi),qa), a)) G 
Kfi for A-a.e. r G [a,b)j and for every a G [0, oq] (see Remark 28). For every a G (0,ao] and 
every t G [r, cr5‘(r)]T, we define 


en{t,a) 


q{t,un{-,a),qa) - q{,t,u,qa) ^ ^ 

———^ - hYi{t,U,qa) 

a 


It suffices to prove that en(’,<a) converges uniformly to 0 on [r, (T*(r)]T as a tends to 0. For every 
a G (0, tto], since the function en(’, <a) vanishes at t = r and is absolutely continuous on [r, cr*(r)]T, 
en(t, a) = e^(r, a) At for every t G [r, cr)‘(r)]T, where 




/(r, g(r, un{-,a),qa), u{r) + a{y - u{r))) - /(r, g(T, u, qa),u{r)) 

a 

df df 

- -^{T,q{T,u,qa),u{r))hn{T,u,qa) - ■^{T,q{T,u,qa),u{r)) {y - u{r)), 


for A-a.e. r G [r, CT*(r))T. Using the Taylor formula with integral remainder, we get 

df 

£n (t, a) = / ^ (*ear) d9 eu (t, a) 

Jo oq 

^ df df \ 

J -^{*ear)d9 - —{T,q{T,u,qa),u{r))\ hu{T,U,qa) 

^ df df \ 

J ^{*eaT)d 9 - —{T,q{T,u,qa),u{r))\ {y-u{r)), 


where 


*eaT = (r, q(T, u, qa) + 9{q{T, un(-, a), qa) - qir, u, qa)), u{r) + 9a{y - u{r))) G Kr. 
It follows that ||£^(r,a)||B" < xn(r,a) -1-Li?||£n(r, a)||Rr., where 


Xn(r,a) = 


Therefore, one has 


df df \ 

J -^{-k9aT)d9 -—{T,q{T,u,qa),u{r))j hniT,U,qa) 
(/ {y - u{r)) 


||en(t,a)||R" < / xn{T,a) At + Lr ||£n(r, a)||Rr. At, 

J[r,a-l{r))t d [r,t)t 

for every t G [r,CT*(r)]T. From Lemma 6, ||£n(t, Q!)||r" < TI'n(Q!)ei^(CT*(r),r), for every t G 
[r,cr)‘(r)]T, where Tn(a) = Xn(T,a) At. 

To conclude, it remains to prove that Tn(Qf) converges to 0 as a tends to 0. Since q{T, un(’, <a), qa) 
converges uniformly to q{T,u,qa) on [r, crJ(r)]T as a tends to 0 (see Lemma 9) and since df/dq 
and df /du are uniformly continuous on Kr, the conclusion follows. □ 
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Then, we define the so-called second variation vector wyi{‘-,u, qa) associated with the needle-like 
variation If = (r, y) as the unique solution on {(Ji{r), 6 ]t of the linear A-Cauchy problem 

w^it) = ^{t,q{t,u,qa),u'^it))w{t), w{al{r)) = hn{(TUr),u,qa). (17) 

The existence and uniqueness of wn{-,u,qa) are ensured by [15, Theorem 3]. 

Proposition 3. The mapping 

Fn{u,qa): ([0,ao],|-|) ^ (C(K(r), 6]t,M"), || • ||oo) 

a I—!• g(-,un(-,a),go) 


is differentiable at 0, and one has DFuiu, qa){0) = wu{‘,u,qa)- 

Proof. We use the notations of the proof of Lemma 8. From Proposition 2, the case cr['(r) = 6 is 
already proved. As a consequence, we only focus here on the case cr[‘(r) = tTi(r) < b. 

Recall that {T,q{T,uu{-,a),qa),uf^{'r,a)) G Kn for A-a.e. r G [a, 6 )t and for every a G [0, oq] 
(see Remark 28). For every a G (0, oq] and every t G [cri(r), 6]t, we define 


sn(t,a) 


q(t,un(-,a),qa) - q(t,u,qa) ,, . 

-wn(t,u,qa) 

a 


It suffices to prove that £n(-,Q;) converges uniformly to 0 on [cri(r),6]T as a tends to 0. For every 
a G (0, oq], since the function £n(’,a) is absolutely continuous on [(Ti(r),5]T, we have en(t,a) = 
en(CTi(r),a) At for every t G [cri(r),6]T, where 


£n(A 'a) — 


/(r, q(T, un(-, a), qa), u'^(t)) - /(r, q(T, u, qa),u'^(T)) 


df, 


- ^(A q{T, u, qa), U^{t)) Wnij, U, qa), 


for A-a.e. r G [CTi(r),6)T. Using the Taylor formula with integral remainder, we get 



qi 

dq 


{*ear) dO enir, a) 


+ 



{Foct ) do 


qi 

dq 


{T,q{T,u,qa),u^{T)) 


Wll{T,U,qa), 


where: 

= (t, q{T, U, qa) + 0{q{T, Un{-, a), qa) - q{T, U, qa)), U^{t)) G Kr. 
It follows that ||£^(T,a)||E" < xn{T,a) -I-LE||£n(T, a)||R.*, where 

pi 

Xn(T,Q;) = 

Therefore, one has 


/ 9f 9f \ 

J -^{*dar)dd - -^{T,q{T,U,qa),u‘^{T))j Wn{T,U,qa) 


kn(i,Q!)|| r" < ||£n(o-i(r),a)||R"-I- / xniT,a) At + Lr ||£n(T,a) 

J[<Ti{r),b)i J[<7i{r),t)r 


At, 


for every t G [ai{r),b]f. It follows from Lemma 6 that ||£n(i, <a)||B" < ^ii{o:)eLj^{b,ai{r)), for 
every t G [cri(r),6]T, where Tn(Q:) = ||£n(cri(r), q;)||r- -Fxn(r, a) At. 


24 








To conclude, it remains to prove that Tn(Q!) converges to 0 as a tends to 0. First, from 
Proposition 2, it is easy to see that ||en(o'i(?’), a)||Rr. converges to 0 as a tends to 0. Second, since 
( 7 (r, un(-, a), qa) converges uniformly to q(T, u, qa) on [cri(r), b]j as a tends to 0 (see Lemma 9) and 
since df/dq is uniformly continuous on we infer that b)T At converges to 0 as 

a tends to 0. The conclusion follows. □ 


Lemma 10. Let R > ||M||L“([a,6)T^,H"*) let {uk,qa,k)keK be a sequence of elements ofRiii{u,qa). 
If Uk converges to u Ai-a.e. on [a,5)Ti and qa,k converges to qa in M" as k tends to +oo, then 
hu{-,Uk,qa,k) converges uniformly to hYi{-,u,qa) on [r, CT*(r)]T as k tends to +oo. 


Proof. We use the notations Kn, and rjn, defined in Lemma 5 and in its proof. 

Let us consider the absolutely continuous function defined by efc(-) = hu{-,Uk, qa,k) — hu{-,u, qa) 
on [r, cri{r)]T. Let us prove that Sk converges uniformly to 0 on [r, cri(r)]T as k tends to +oo. Since 
£kir) = 0, one has 

f df 

^k{t) = / —(r, g(T,Mfc,qa.fe),Mfe(r))£fe(r) Ar 

f f ^ f ^ f '\ 

+ ^ y-^iT,q{T,Uk,qa,k),Ukir))--^{T,q{T,u,qa),u{r))jhu{T,u,qa)AT 

[-^{T,q{T,Uk,qa,k),Uk{r)) - ■^{T,q{T,u,qa),u{r))j {y - u{r)) At 

f df 

+ / -^{'r,q{T,Uk,qa,k),Uk{r)){u{r) - Uk{r)) At 

J\T.f)-r OU 


for every t € [r,cr*(r)]T and every fc G N. Since {uk,qa,k) G Ei^(w,(Za) for every fc G N, it follows 
from Remark 28 that {T,q{T,Uk,qa,k),uf{T)) G Kr for A-a.e. t G [a,5 )t. Hence it follows from 
Lemma 6 that ||£fc(t)||R’i < TfceL^(6,r), for every t G [r, cr*(r)]T, where is given by 




LR^ll{r)\\u{r) 



fU 

dq 

dl 

du 


- ■Ufe(r)||R™ 

(T,g(T, Uk,qa,k),Uk{r)) 

(r, g(T, Uk,qa,k),Ukir)) 


df 

— {T,q{T,u,qa),u{r)) 


dl 

du 


{r,q{T,u,qa),u{r)) 


||/in('r,u,qa)||R" At 
\\ y - u(r)||Rr. At. 


Since /rAi({r}) = A^i(r) > 0, Uk{r) converges to u{r) as k tends to +oo. Moreover, from the 
Lebesgue dominated convergence theorem, {uk,qa,k) converges to (u,qa) in (Er(u, ga), || • WuQb) 
and, from Lemma 7, q{-,Uk,qa,k) converges uniformly to q{-,u,qa) on [o, 6 ]t as k tends to +oo. 
Since df/dq and df /du are uniformly continuous on IGr, we conclude that converges to 0 as 
k tends to +oo. The lemma follows. □ 


Lemma 11. Let R > ||w||L.j°([a,h)T,^,R"*) and let {uk,qa,k)ken be a sequence of elements ofRiii(u,qa). 
If Uk converges to u Ai-a.e. on [a, 5 )ti and qa,k converges to qa in M" as k tends to +oo, then 
wni‘,Uk, qa,k) converges uniformly to wn{-,u,qa) on [crJ(r),6]T as k tends to +oo. 

Remark 29. With the same arguments of the proof of Lemma 4, the hypotheses of Lemma 11 
give the convergences of u^ to u^ A-a.e. on [a, h)f and, from the Lebesgue dominated convergence 
theorem, of {uk,qa,k) to (m, ga) in (Er(m, g^), || • LcO- 
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Proof of Lemma 11. We use the notations Kji, Ljj, vji and defined in Lemma 5 and in its 
proof. From Lemma 10, the case a* (r) = 5 is already proved. As a consequence, we only focus 
here on the case crl{r) = ai{r) < b. 

Let us consider the absolutely continuous function defined by efc(-) = u>n(-, Uk, qa,k)—wii{‘,u, qa) 
on [cri(r),6]T. Let us prove that Sk converges uniformly to 0 on [cri(r),6]T as k tends to +oo. One 
has 

ek{t) = eki<7i{r)) + / -^{T,q{T,Uk,qa,k),ut{T)) Skir) At 

J[ai{r),t)i oq 

+ [ (^ir,q{T,Uk,qa,k),ut{T))-^{T,q{T,u,qa),u‘^{T))] wn{T,u,qa) At, 

J[cri(r),t)T \Oq oq J 


for every t G [cri(r),6]T and every fc G N. Since {uk,qa,k) G Efi{u,qa) for every fc G N, it follows 
from Remark 28 that {T,q{T,Uk,qa,k),Uk{T)) G Kr for A-a.e. r G [cri(r),6)T. Hence it follows 
from Lemma 6 that 

IkfeWIU" < (efc(o-iW) + (6,0-1 (r)), 

for every t G [o-i(r),6]T, where T/j is given by 


Tfc = 


'[dri(r),b)T 


^{T,q{T,Uk,qa,k),ut{T)) - ^{T,q{T,U,qa),U^{T)) 


lkn(T,u,qo)||B" At. 


Since {uk,qa,k) converges to {u,qa) in (Eii{u,qa), IMImqO and from Lemma 7, q{-,Uk,qa,k) converges 
uniformly to q{-,u,qa) on [a, 6]t as k tends to +oo. Since df/dq is continuous and bounded 
on Kfi and since rtf' converges to A-a.e. on [a, 6)t, the Lebesgue dominated convergence 
theorem concludes that converges to 0 as /c tends to -l-oo. Finally, from Lemma 10, one has 
£fc(CTi(r)) = wn(cri(r),Ufe,ga,fc) - wn(CTi(r),u,9a) = hYi{(Ji{r),Uk,qa,k) - hn{(yi{r),u,qa) converges 
to 0 as /c tends to -l-oo. The lemma follows. □ 


4.2.3 Needle-like variation of m at a point s G RDi 

Let s G (7(-, -u, (7a), w^)) H RDi and z G M™. Note that s G Ti and then <i>(s) = s. We 

define the needle-like variation H = (s, z) of m at s by 




z if t G [s,s-f/3 )ti, 

u{f) if t ^ [s,s-f/3 )ti. 


for Ai-a.e. t G [a, 6 )ti and for every /3 G Vi’^ C V®’^. 

Lemma 12. There exists /3o > 0 such that (uu(-,/?),<Za) G UQ^ for every fi G Vi’^ n [0,/3o]. 

Proof Let R = max(||w||L~([Q,6)Tj,K™), Iklk™) + 1 > ||u||l“ ([a,b)Tj ,r™) • We use the notations Kr, 
L_r, and ?7ij, defined in the proof of Lemma 5. For every/3 G Vi’^, we have ||uu(-, ,d)||L“ ([a,b)Tj .R™) ^ 
R and 

||uu(-,/3) - m||li ([a.bhi.R-) = [ ||z - M(r)||B-Air < 2i?/3. 

Hence, there exists /3o > 0 such that for every (3 G Vi’** n [0,/3o], IkuC-j/d) — u||li ([a,b)Ti,R™) — 
and thus (uu(-,/3), 9a) G PiR(u,qa). The conclusion then follows from Lemma 5. □ 


26 






Lemma 13. The mapping 

Fu{u,qa)-. (Vi^’'’n[0,/3o],|-|) ^ (C([a,&]T,K”),||-||oo) 

P ' —^ 9(-,uu(-,/3),go) 

is Lipschitzian. In particular, for every /3 G H [0,/3o], ?(•, uu(’i/?); ^a) converges uniformly to 
q{-,u,qa) on [a, 6 ]t as /3 tends to 0. 

Proof. We use the notations of the proof of Lemma 12. From Lemma 7, there exists C > 0 
(Lipschitz constant of Fji{u,qa)) such that 

\\q{-,Uu{-,P 2 ),qa) - q(-,Mu(-,/3l),9a)||oo < C\\{uu{-, P 2 ) , qa) “ iuui', Pi), qa)\\u Q’’ 

< 2CR\P2-PiI 

for all Pi and P 2 in H [0,/3o]- The lemma follows. □ 

According to [15, Theorem 3], we define the variation vector wii{‘,u,qa) associated with the 
needle-like variation If = (s, z) as the unique solution on [s, 6 ]t of the linear A-Cauchy problem 

Q / 

w^{t) = -^{t,q{t,u,qa),u'^{t))w{t), w{s) = f{s,q{s,u,qa),z) - f{s,q{s,u,qa),u{s)). ( 18 ) 

Proposition 4. For every S G V®’^\{0}, the mapping 

Fu{u,qa): (Vi^’'n[0,/3o],|-|) ^ (C([s + <5,5 ]t,M"), || • ||oo) 

P ' -5- q{-,Uu{-,P),qa) 


is differentiable at 0 , and one has DFii{u,qa){0) = wui-,u,qa). 

Proof. We use the notations of the proof of Lemma 12. Recall that {T,q{T,uu{-, P), qa),Uu{T, P)) 
and {t, q{T, uu{-, P), qa) , z) belong to Kfi for every /3 G Vi fl [0,,5o] and for A-a.e. r G [a, 6 )t (see 
Remark 28). For every /3 G n (0, Pq] and every t G [s -k /3, 5 ]t, we define 

/, q{'t,Uu{-,P),qa) - q(t,U,qa) ,, . 

eii{t,P) = -^- wii{t,u,qa). 


It suffices to prove that eu(-, /3) converges uniformly to 0 on [s -k /3, 6 ]t as P tends to 0. Note that, 
for every 5 G V'^’^\{0}, it suffices to consider P < 5. For every P G n (O,,0o], the function 
eu(’) P) is absolutely continuous on [s -k P, 6 ]t and £u(t, P) = £u(s -k P, P) (t,/3) At, 

for every t G [s -k /3, &]t, where 


£u(a/3) = 


fir, q{T, uu{-, P),qa),u'^i.T)) - fij, qij, u, qa), u'^{t)) 


P 


df 

- g(r, u, qa), u'^(t)) Wn(T, u, qa). 


for A-a.e. r G [s -k /3, 5 )t- Using the Taylor formula with integral remainder, we get 
£u(A/ 3 )=y ^{*epr)d0eu{T,P) 

+ ij -^i*9pT)de - -^{T,q{T,U,qa),u'^{T))] WuiT,U,qa), 
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where: 


*e/3r = (t, q{T, U, Qa) + 9{q{T, Uu{-, 13), qa) - q{T, U, qa)),U^{T)) e Kr. 

It follows that ||e^(r,/3)||Rr. < xu('r,/3) + Ln\\eu{T, P)\\Rr,, where 

Xu(r,/3) = 

Therefore, one has 

||£u(t,/3)||K" < ||eu(s + /3,^)||R" + / xu{t, 13) At + Lr f ||eu(T,/3)||K" At, 

J[s+/3,b)t J[s+j3,t)t 

for every t G [s + /3, 61 t, and it follows from Lemma 6 that ||eu(i, ;d)llR’* < Tu(/3)ei„ (6, s), for every 
tG [s + /3,6]t, where Tu(/3) = ||eu(s +/3,/3 )||b" +Xu(r,/3) At. 

To conclude, it remains to prove that Tu(/3) converges to 0 as tends to 0. Since q{-, mu(-, 13), qa) 
converges uniformly to q{-,u,qa) on [s + /3, 6 ]t as (3 tends to 0 (see Lemma 13) and since df/dq 
is uniformly continuous on Kr, we first infer that Xu('r, /3) At converges to 0 as /3 tends 

to 0. Secondly, let us prove that ||eu(s + /3 ,/?)||k" converges to 0 as /? tends to 0. By continuity, 
u>n(s + f3,u,qa) converges to wu{s,u,qa) as f3 to 0. Moreover, since g(-, ttu(-,/?), 9 a) converges 
uniformly to q(-,u,qa) on [a, &]t as j3 tends to 0 and since / is uniformly continuous on Kr, it 
follows that /(•, (?(•, Mu(-, /3), 9 a)) z) converges uniformly to /(•, q{-, u, qa), z) on [a, &]t as (3 tends to 
0. Therefore, it suffices to note that 

4 f {f{T,qiT,u,qa),z) - f{T,q{T,u,qa),u‘^{T))) At 
P j[s,s+/9)t 

converges to wu(s, u, qa) = f{s, q{s, u, qa), z) — f{s, q{s, u, qa),u{s)) as {3 tends to 0 since s is a A- 
Lebesgue point of f{-,q{-,u,qa),z) by continuity and of f{-,q{-,u,qa),u^{-)) by hypothesis. Then 
||en(s + j3,j3)\\Rr, converges to 0 as /3 tends to 0 , and hence Tu(/3) converges to 0 as well. □ 

Lemma 14. Let R > ||M||L.j°([a,h)Tj ,K"*) and let (uk,qa,k)kGH be a sequence of elements ofRiR{u,qa)- 
Ifuk eonverges to u Ai-a.e. on [a,6)Ti, Uk{s) converges to u(s) and qa,k converges to qa as k tends 
to +(X), then wui‘,Uk,qa,k) converges uniformly to wu{-,u,qa) on [s,&]t as k tends to +oo. 

Proof. The proof is similar to the one of Lemma 11, replacing ai{r) with s. □ 

4.2.4 Variation of the initial condition qa 

Let 9 (, G K". 

Lemma 15. There exists 70 > 0 sueh that {u,qa + 79 ^) GUQ^ for every 7 G [ 0 , 7 o]. 

Proof Let R = ||M||L“([a,b)Ti.R’") + l > ll'a||L“([a,b)Ti,K™)- We use the notations Kr, Lr, vr and tjr, 
defined in the proof of Lemma 5 . There exists 70 > 0 such that ||9a + 79(1 — 9a||R" = 7 l|<?al|H" A VR 
for every 7 G [ 0 ,70], and hence {u, 9 a+ 79 a) G ER(n, qa). Then the claim follows from Lemma 5 . □ 

Lemma 16. The mapping 


(/ ^(*®dr)d6' - |^(T,9(T,M,9a),n‘^(T))^ Wu{T,U,qa) 


F,,^{u,qa): ([0,7o],|-|) (C([a,6]T,K"),||-||oo) 

7 I—^ q{-,u,qa + jqa) 

is Lipschitzian. Ln particular, for every 7 G [0, 70 ], q{-, u, 9a + 79 a) eonverges uniformly to q{-,u, qa) 
on [a, 6 ]t as 7 tends to 0 . 
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Proof. We use the notations of the proof of Lemma 15. From Lemma 7, there exists C > 0 
(Lipschitz constant of Fji{u,qa)) such that 


||g(-,u,(7a +7291) - 9(-,M,9a +7i9l)l|oo < C\\{u,qa+"f 2 qa) - (u,9a + 7i9l)liwe'> 

= (^172 -7illklllB"- 


for all 7 i and 72 in [ 0 , 7 o]. □ 

According to [15, Theorem 3], we define the variation vector Wq> {■,u,qa) associated with the 
perturbation q[^ as the unique solution on [a, 6 ]t of the linear A-Cauchy problem 

Of 

w^{t) = -^it,q{t,u,qa),u‘^it))w{t), w{a) = q'^. (19) 

Proposition 5 . The mapping 


■■ ([0,7o],|-|) —^ (C([a,6]T,K"), II • Iloo) 

7 '— <l{-,u,qa + iq'a) 

is differentiable at 0 , and one has DFq>^{u,qa){ 0 ) = Wq'^{-,u,qa). 

Proof. We use the notations of the proof of Lemma 15. Note that, from Remark 28, (r, g(r, u, qa + 
'yq'g^),u'^{ t)) G Kh for every 7 G [ 0 , 7 o] and for A-a.e. r G [a, 6 )t. For every 7 G ( 0 , 7 o] and every 
t G [a, 6 ]t, we define 


f, N qit,u,qa+'fqa) - qit,u,qa) ,, . 

7 

It suffices to prove that £q^(-, 7 ) converges uniformly to 0 on [a, 6 ]t as 7 tends to 0. For every 
7 G ( 0 , 7 o], since the function eq> (-, 7 ) vanishes at t = a and is absolutely continuous on [a, 6 ]t, 
^9' (^> 7) = /[a,t)T 7) Ar, for every t G [a, 6 ]t, where 


^<(+ 7 ) = 


/(r,g(T,u,ga +79l),'w ("t)) - /(t, 9(t, u, g^), u®(t)) 


7 


df 

- -^{r, g(r, u, qa), u‘®’(r), r) Wq>^{T, u, qa), 


for A-a.e. r G [a, b)j. Using the Taylor formula with integral remainder, we get 
^<(A7)=^ ^i-* *^Sjr)d0 ■ eq>^{T,'y) 

+ (^J ^{*e'yT)d9- ^(r,g(T,M,ga),u‘®’(T))^ Wq>^(T,U,qa), 


where 

*e-tT = (r, g(r, u, qa) + 6 >(g(r, u, qa + 79I) - ?(+ u, qa)), u‘^(t)) G Kr. 
It follows that ||ef^(T, 7 )||B’* < X 9 ^(t, 7 ) + LB||eq^(T, 7 )||B", where 


X 9 '(u7) = 




-fr)d0- ^{T,q{T, 


qa),U (t)) Wq>^(T,U,qa) 
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Hence 

lie?'(i>7)l|R" < / Xg'(T, 7 )AT + Lij /" Ije,/(r, 7 )||Kr. At, 

for every t G [a,5]T, and it follows from Lemma 6 that \\eq> (t, 7 )||B’* ^ Tf?' {l)^La{b^o)i for every 
t G [a, &]t, where T,/ ( 7 ) = (r, 7 ) Ar. 

To conclude, it remains to prove that Tg^ ( 7 ) converges to 0 as 7 tends to 0. Since q{-,u, qa+iq'a) 
converges uniformly to q{-,u,qa) on [ 0 , 6 ]^ as 7 tends to 0 (see Lemma 16) and since df/dq is 
uniformly continuous on Kn, the conclusion follows. □ 

Lemma 17. Let R > ||u||L“([a,b)T,R™) iuk,qa,k)keN be a sequence of elements o/Eb(m, ga). 

If Uk converges to u Ai-a.e. on [o, 5 )ti and qa,k converges to qa in K” as k tends to +00, then 
Wq'^{-,UkT qa,k) converges uniformly to Wqr^{-,u,qa) on [o, 6 ]t as k tends to +00. 

Proof. The proof is similar to the one of Lemma 11, replacing cri(r) with a. □ 

4.3 Proof of Theorem 3 

We are now in a position to prove the PMP. In the sequel, we consider q* an optimal trajectory, 
associated with an optimal sampled-data control u* and with b* G T, with 6 * = 6 if the final time 
is fixed. We set 9 * = q*{a). 

4.3.1 The augmented system 

As in [43, 46], we consider the augmented system in 

= f{t,q{t),u‘^{t)), (20) 

with q = ( 9 ,( 7 °)^ the augmented state with values in K” x M, and / : T x x K™ —>■ 

the augmented dynamics, defined by f{t,q,u) = {f{t,q,u),f^{t,q,u))^. Note that / 
does not depend on We will always impose as an initial condition < 7 °(a) = 0, so that 
Q°{b) = J[a b)r '?('^)> '“*('^)) Hence, the additional coordinate stands for the cost. 

Denoting g* = (q*, 0)T, we have iu*,q*) G UQ^ . We set q* = q{-,u*,ql) = {q\q^*y. 
Hence, q* is a solution of (20) on [a, 5*]t associated with the control u*, satisfying g°*(a) = 0 
and g{q*{a),q*{b*)) G S and minimizing q°{b) over all possible trajectories q solutions of (20) on 
[a, 6]t, where & G T and associated with a control u G L|P ([a, &)ti , D), satisfying g°(a) = 0 and 
g{q{a),q{b)) G S. 

4.3.2 Application of the Ekeland variational principle 

For the completeness, we recall a simplified (but sufficient) version of the Ekeland variational 
principle. 

Theorem 4 ([26]). Let (E, cIe) be a complete metric space and let J : E —)■ IR+, A i-G J(A) be a 
continuous nonnegative mapping. Let £ > 0 and A* G E such that J(A*) < s. Then, there exists 
Ag G E such that d^{X,;,\*) < ^/e and, —^d^{X,Xg) < J(A) — J{Xg), for every A G E. 

Let R > ||u*||l“([o. 6 *)ti,R"')- Hecall that Eji{u*,ql) C UQ^ (see Lemma 5). We set 
Efl°(u*,g*) = {iu,qa) G F.R{u*,ql) \ u G L])P ([a, 6 *)ti , D), fa = (ga,0)}. 
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Note that {u*,ql) G E^’°(u*,g*). Since is closed, it follows from the (partial) converse of 
the Lebesgue dominated convergence theorem that q*), || • is a closed subset of 

L^^([a, X K"’+^ and then is a complete metric space. For every e > 0, we define the 

functional Jf : || • K+ by 


J^{u,qa)= U{q°{b*,u,qa)-q°*{b*)+e)'^^ + dj {g {qa,q{b*,u,qa))) 


1/2 


Since g and dg are continuous and so is Ffi{u*, g*) (see Lemma 7), it follows that is continuous 
on (E^’°('u*,g*), II • WjjQh*)- Moreover, one has J^{u*,ql) = e and, from optimality of g°*(6*), 
J^{u,qa) > 0 for every {u,qa) G E^’°(M*,g*). It follows from the Ekeland variational principle 


that, for every e > 0, there exists (wf, q^^) G E^’^(u*,q*) such that 
and 


(U^,^a,e)-iu*,q*a 


Lc- < 


-^/e\\{u,qa) - {u^,qa,e)\\uQ>’-‘ ^ ^ei^^qa) - 


( 21 ) 


for every {u,qa) G E^’°(u*, g*). In particular, uf converges to u* in ([a, 6 *)ti , R™) and 
converges to q* as e tends to 0. Besides, setting 




-1 






( 22 ) 


and 


€ = 


-1 




{9{qte^q{b* 




)) -Ps{g{qa,e^qib* 


(23) 


note that + ||i/'f||^, = 1 and G C>s[Ps( 5 ( 9 ^e,'?(&*. uf, g^J))]. 

Using a compactness argument, the continuity of Ffi{u*,q*^) (see Lemma 7), the C^-regularity 
of g and the (partial) converse of the Lebesgue dominated convergence theorem, we infer that 
there exists a sequence (e/c)fcgN of positive real numbers converging to 0 such that converges to 
u* Ai-a.e. on [a, 6 *)Ti, converges to ql, 3 ( 9 ^^^, g( 6 *, J) converges to g{ql,q*{b*)) G 

S, converges to <lg{ql,q*{b*)), converges to some 1 /'°^ < 0 , and 

converges to some G R-^ as k tends to + 00 , with |i/;°-^p + ||i/’'^||^j = 1 and —ijj^ G 
Os[g{q*a,q*{b*))] (see Lemma 1). 

In the next lemmas, we use the inequality (21) respectively with needle-like variations of at 
right-scattered points of Ti and at right-dense points of Ti, and then variations of qae^ ■ Hsnce, we 
infer some important inequalities by taking the limit in k. Note that these variations were defined 
in Section 4.2 for any dynamics /, and that we apply them here to the augmented system (20), 
associated with the augmented dynamics /. 


Lemma 18 . For every r G [a, 6 *)ti H RSi and every y G D^g^g(u*(r)), considering the needle-like 
variation 11 = (r, y) as defined in Section 4-2.2, one has 


^°^wUb*,U*,q: 




il)^,wn{b*,u ,ql) 


< 0 , 


(24) 


where the variation vector wn = (li'n, Wj\)^ is defined by (17) (replacing f with f). 

Proof. Since converges to u* Ai-a.e. on [a, 5 *)ti, it follows that u^(r) converges to u*{r) as 
k tends to -too, where ||u*(i’)||r™ < R- It follows that ||Mf^(r)||Rm < R and 0 is not isolated in 
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^ = {a € [0,1], u^^{r) + a{y — uf^[r)) G ft} for any sufScently large k, see Definition 1. Fixing 
such a large k, one has ^ ([a, &*)ti , D) and 

lhf,,n(-:a)llL“([a,f,*)Ti,B’") < max(||Mfj|L~([a,b*)Ti.M-): l|wf,,n(^:a)lk™) 

< max(i?, llufjOllR™ +a||y-Mfjr)||R™), 


for every a G S. Moreover, one has 

Wfj.R ( „,*ll II-11-R 


n(-,a) - M*||Li^([a,b*)Ti.R"') ^ lkefc,n(’>Q;) - Me^||Li^([a,6*)Ti,R™) + \Wek ~ II([a,b*)Ti .R™) 

< ani{r)\\y - uf^(r)||Rr. + 

Therefore jj(-, a), G E^’°(u*, q*) for every a G S sufficiently small and every k sufficiently 

large. It then follows from (21) that 


■\/^ lk£fc,n(G '^efc IIl.J,^ ([a,b*)T^ ,R™) — ,n(G j 9 a,efc ) kj;, j 9 a,efc )i 


and thus 


-v^A^iWlly-<WI|R™ < ,t n \ , R ^ R r \v 


Using Proposition 3, we infer that 


lim 

a—>-0 


«,n(Ga).9LJ" - 


+ ‘^i9{(la,eu > 9{b *, uf^ , J) - Ps {9{qa,eu ^9ib*, wf, , qa,ej)) > 


dq2 


(qt, ’ “f. > 9a,Ck))wnib*,ul, g5^). 


Since qa sk^ converges to e^) tends to 0, using (22) and (23) it 

follows that 






By letting k tend to +oo and using Lemma 11, the lemma follows. 
We define the sets 




* r>.R ^R 


□ 


A = {t G [a, 6 *)ti I does not converge tO'u*(f) whenfctends to + oo} , 

= {t G [a,6*)Ti I t ^ if[a,b.)k/(-,g(-,uf^,g^fc),ufJ^))} , fc G N. 

Lemma 19. IFe have y,Ai U |J^,gj^ A^) = 0. 

Proof. Since /iAi(^) = 0, it suffices to prove that ^Ai(^fc) = 0 for every fc G N. Let fc G N. We set 
Bk = {tG [a,b*)T I t i if[„_i,.)^(/(-,g(-,uf^,g^fc),uff))} . 
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We know that fJ,A{Bk) = 0. Hence, Bk C RD and consequently, fJ.A{Bk) = ^J■L{Bk) = 0. Since 
Ak C Bk, HkiAk) = 0. To conclude, it suffices to prove that fiAi{Ak) = f^kiAk)- To see this, let 
us prove that Ak C RDi. By contradiction, let us assume that there exists t G Ak r\ RSi. Since 
t £ Ak C RD, we conclude that is constant on [t, min((Ti(t), 5*))t, where (t, min(tTi(t), 6*))t ^ 
0. As a consequence, f {■ , q{-, k)^ is continuous on [t, min(cri(t), 5*))t and consequently 

t € This leads to a contradiction. □ 

We define the set of Lebesgue times t G [a, 6*)ti by 

■^SU = i^^b*)rA(Au[jAk). 

\ feGN / 

Note that fJ.Aii-^[a’b-)t'^ = MAi([a,^*)Ti) and then ^J,A^{^[a'^k•)r = MAi([a, &*)ti H RDi). 

Lemma 20. For every s G ri RDi and for every z G id D BRm(0, i?), considering the 

needle-like variation H = (s,z) as defined in Section 4-2.3, one has 


where the variation vector tcu = (wn,rcu)^ is defined by (18) (replacing f with f). 

Proof. For every k gN and any (3 G , we recall that jj(-,,d) S L|P ([a, &*)ti , D) and 

ll“ffc,u(’;/3)llL”([a,b*)Ti,K'^) < IUaxdjuf^ || ,R™): INIIm"*) ^ 

and 

ll^ffe.u(’)/^) ~ ■a*||Li^([a,b*)Ti.R’") ^ /3) — ^^ffcllLi^([a,b*)Ti,R’") + W^e'k ~ ■“* IIli^ ([a,b*)Ti .R™) 

< 2RP + ^/sk- 

Therefore {uf^ u('> z®)’^ every /3 sufficiently small and every k sufficiently 

large. It then follows from (21) that 


-V^IIW.,u(-,/ 5 ) - wjlu ([a,b*V.R-) < JeMe,,uiGP),qa,eJ " (W.: 9 a.£ J, 


and thus 


—2R-^/ek Si 


.U (•- /^) z qa,e. ) + : qle.)) ' 


Using Proposition 4, we infer that 


lim 

/ 3 -s.O 






= - g°*(&*) + 

+ 2(ff (g^-e > ’ €,ej) - Ps {9{qa,eu ’ 

^ {qa,eu ^9{b*, uf, , 95 J) ^U{b*, , 95 j) j 
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Since y(-, I3),qa^j converges to as /? tends to 0, using (22) and (23) it follows 

that 


By letting k tend to +oo, and using Lemma 14, the lemma follows. 


□ 


Lemma 21. For every qa € K" x {0}, considering the variation of initial point as defined in 
Section 4-2.4, one has 


where the variation vector Wq^ = (wq^,Wq^)^ is defined by (19) (replacing f with f). 
Proof. For every fc G N and every 7 > 0, one has 


(26) 


\Qa.e, + 79a - 9 II|r"+i < 7ll9a||R"+i + - g(;||H"+i 


< 7ll9a||R-+i + 


^a,6k 

Therefore + 79 a) G E^’°(u*,g*) for every 7 sufficiently small and every k sufficiently 

large. It then follows from ( 21 ) that 


+79a - 9"eJlR-+i < ,9"e^ +79a) - C,eJ ^ 


and thus 


-V^ha 


.+1 < 


■'.iK'j.g.. +Tg.>" - 

li-’tMt.-il,, + w-) + 


Using Proposition 5, we infer that 


■'gK.g.. +7?.)" - 

7—>0 7 

= 2{q°{b*, ) - q°*(6*) + Efc)u;°J6*, ) 

+ 2(9(9^^e,, 9(fe*, , 9^ J) - Ps ( 9 ( 9 ^^^,,, 9(&*, wf,, 9^^£ J)), 


dg 

dqi 


(95. > 9 (&* , < , 95 J) 9 a + ^ (95. ’ 9 (&* , < , 95 J) (&* : < > 95 j)^, • 


dq2 


Since (uf^, q(f ,^^ + jqa) converges to (uf^, ) as 7 tends to 0, using (22) and (23) it follows 

that 

- V^ll9a||R"+i < (^(9^!^. > 9(&*, 9^ J)) 

- (^{§^ida,e,^Qib%U^,,qa,eJ)) , 9^ J ' 

By letting k tend to +cx), and using Lemma 17, the lemma follows. □ 
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At this step, we have obtained in the previous lemmas the three inequalities (24), (25) and (26), 
valuable for any R > ||m*||l“([ a,6 )ti,»"*)• Recall that = 1 and that G 

Os[g{qa,q*{b*))]- Then, considering a sequence of real numbers Ri converging to +oo as £ tends 
to +00, we infer that there exist < 0 and ^ G W such that converges to 'ijP and '0'^^ 

converges to 0 as £ tends to +oo, and moreover |0°P + ||0||gj = 1 and —0 G Os[g{ql,q*{b*))]. 
We set = n^GN-^[a,b*h- Ai([o, 6 *)ti) and consequently 

n RDi) = fj.Ai{[a,b*)j^ n RDi). Taking the limit in i in (24), (25) and (26), we get 
the following lemma. 

Lemma 22 . For every r G [a, &*)ti H RSi, and every y G D^„^|3(u*(r)), one has 



where the variation vector wn = (rcn,iCn)^ associated with the needle-like variation 11 = {r,y) of 
u is defined by (17) (replacing f with f). 

For every s G n RDi and every z G fl, one has 



where the variation vector wn = (wu,'Wn)^ associated with the needle-like variation If = (s, z) of 
u is defined by (18) (replacing f with f); 

For every fa G K" x {0}, one has 



where the variation vector Wq^ = associated with the variation Pa of the initial point 

q* is defined by (19) (replacing f with f). 


This result concludes the application of the Ekeland variational principle. The last step of the 
proof consists of deriving the PMP from these inequalities. 

4.3.3 Proof of Remark 11 

In this subsection, we prove the formulation of the PMP mentioned in Remark 11. Note that we 
do not prove, at this step, the transversality condition on the final time. 

We define p = {p, p°)^ as the unique solution on [a, 6 *]t of the backward shifted linear A-Cauchy 
problem 


p'"{t), pib*) = (j^^{ql,q*{b*))^ 0,0°^ . (30) 

The existence and uniqueness of p are ensured by [15, Theorem 6 ]. Since / does not depend on 5 °, 
it is clear that is constant with = 0 ° < 0 . 
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Right-dense points. Let s S flRDi and z £ fl. Since the function {p, ■icu(’, 9a))R'*+i 

is absolutely continuous and satisfies (p, t(;u(’,u*,do))Mn+i = 0 A-a.e. on from the Leibniz 

formula (3), this function is constant on It thus follows from (28) that 

(p(s),-u;n(s,M*,g*))R«+i = (p(6*),iZ;u(&*,M*,g*))R«+i 

= 'tp°w^{b*,u*,q*^) +(^(^^{ql,q*{b*))'^ tp,wu{b* ,u* ,q*j'^ <0, 

and since wu(s, = f{s,q*{s),z) - f{s,q*{s),u*{s)), we finally get 

{p{s)J{s,q*{s),z))Rr.+i < (p(s),/(s,g*(s),'U*(s)))R.+i. 

Since this inequality holds for every z £ fl, we have obtained the maximization condition 
H{s,q*{s),p{s),p°,u*{s)) = maxH{s,q*{s),p{s),p°,z). 

zGQ, 


Right-scattered points. Let re [a, 6*)TinRSi andy e D^ 3 ^|^(u*(r)). Since (p, wn(-, m*, (7*))r"+i 
is absolutely continuous and satisfies {p, wn{-,u*,ql))^^+i = 0 A-a.e. on [a^ (r), 6 *)t from the Leib¬ 
niz formula (3), this function is constant on [cr*(r), 6 *]t. It thus follows from (27) that 

(p(cr*(r)),u)n(o-)'(r),M*,g;))R„+i = {p{b*),wn{b* ,u* 

= tl}°w^{b*,u*,q*J + (^(^-^{ql,q*{b*))^ ip,wn{b* ,u* ,ql)^ < 0 . 

We recall that wn{crl{r),u*,ql) = hu{(J*{r),u*,ql) where the variation vector hu = {hu,h^)^ 
associated with the needle-like variation 11 = (r, y) of u is defined by (16) (replacing / by /). Since 
hii{r,u*,q*) = 0, it follows that 


[r,cr*(r))T 


(p,/in(-,w*,gI))R™+i(T-) Ar = (p(cr*(r)),/in(crt(r),u*,g;))R„+i < 0. 


Using the Leibniz formula (3) in the integrand and using (16) (replacing / by /) and (30), we 
finally get 


'[r,crU^)h 


f)J-T 

-^{T,q*{ t),p'"{t),p°, u*{r)) At , y-u*{r) 


< 0 . 


Transversality conditions. The transversality condition on the adjoint vector p at the final 
time b* has been obtained by definition (note that —ip G Os[g{qa, Q*(b*))] as mentioned previously). 
Let us now establish the transversality condition on the adjoint vector p at the initial time a. Let 
ga € R" X {0}. With the same arguments as before, we prove that the function {p, Wq^ (•, it*, g*))Rn+i 
is constant on [a, &*]t. It thus follows from (29) that 


{p[a),wq^[a,u*,ql))^r.+i = {p{b*),wq^{b*,u*, q*^))^n.+i 

= (&*))) 'fp,wq^{b*,u*,q*)^ 

and since Wq^ (a, u*,q*) = qa = {qa, 0), we finally get 

Since this inequality holds for every G K" x {0}, the left-hand equality of (8) follows. 
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4.3.4 End of the proof 

In this subsection, we conclude the proof of Theorem 3. Note that we prove the fourth item of 
Theorem 3 in Remark 30 below. To conclude the proof of Theorem 3, we will use the result claimed 
in Remark 11. Let us separate two cases. 

Firstly, let us assume that g is submersive at {ql,q*{b*)). In that case, we have just to prove that 
the couple {p,p^) is not trivial. Let us assume that p is trivial. Then p{a) = p{b*) = 0 and, from 
the transversality conditions on the adjoint vector, belongs to the kernels of {dq^g{ql,q*{b*)))^ 
and [dq^gipa, It follows that ■0 belongs to the orthogonal of the image of the differential 

of g at {ql,q*{b*)). Since g is submersive at {ql,q*{b*)), it implies that tp = 0. Since the couple 
is not trivial, we conclude that 7 ^ 0 and then (p,p°) is not trivial. This concludes the 
proof of Theorem 3 in this first case. 

Secondly, let us assume that g is not necessarily submersive at {ql,q*{b*)). In that case, one 
has to note that if q* is an optimal solution of (OSDCP)-J!^ associated with the function g and 
with the closed convex S, then q* is also an optimal solution of (0SDCP)!|[^ associated with the 
function g defined by g{qi,q 2 ) = ( 91 , 92 ) (which is submersive at any point) and with the closed 
convex S = { 9 *} x {g*(5*)}. Then, we get back to the above first case, but with a different function 
g and a different closed convex S that would impact only the third item of Theorem 3. 

Remark 30. Throughout this remark, we assume that all asumptions of the fourth item of The¬ 
orem 3 are satisfied. As mentioned in Remark 5, Theorem 3 can be easily extended (without the 
fourth item, for now) to the framework of dynamical systems on time scales with several sampled- 
data controls with different sets of controlling times. To prove the transversality condition on the 
hnal time, we use the multiscale version (without the fourth item) of Theorem 3. 

Let 5 > 0 be such that [b* — <5, 6 * -|- 5] C T and such that (/, /°) is of class on [b* — S,b* + 
(5] X M” X K™. Obviously, the trajectory q*, associated with b* and u*, is an optimal solution of 
the optimal sampled-data control problem, with free final time b G {b* — S, b* -I- <5), defined by 


(o).._ 


[b* —( 5 , 6 *+ 5 ] 


min f /°(T,g(r),M‘*’(T))dT, 

Jb‘-S 

' <i{t) = f{t,qit),u^{t)), for a.e. tG[b* — 6,b), 
ugl-(Ti,0), 

^ iq{b* - S), giq*{a), 9 ( 6 ))) G { 9 *( 6 * - <5)} x S. 


We set p*{s) = S{s—l) + b*, x*{s) = q* op*(s) and w*{s) = 6, for every s G [0,1]. With the change 
of variable x = qo p and with p = w,\i is clear that the augmented trajectory {p* ,x*), associated 
with the augmented control {w*,u*), is an optimal solution of the optimal sampled-data control 
problem, with fixed final time s = 1, defined by 

/•i 

min / w{T)f[p{T),x(T),u^°P{T))dT, 

JQ 

(p(t), x{t)) = {w{t), w{t)f{p{t),x{t), for a.e. t G [0,1), 

|(u;,u)gL-([0,1],[V2,+oo))xL?((Ti,L!), 
ipi0),xm = (b* - S,q*ib* - 6)), 

^ (p(l), 9 ( 9 *(a), a;(l))) G [b* - 5* -h (5] X S. 


In this new optimal sampled-data control problem, the sampled-data controls w and u are dehned 
on different time scales. Its Hamiltonian H is H{p,x,pp,px,p^,w,u) = PpW + {pxTwf{p,x,u))Kn. + 
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p^wf^(p,x,u). Applying the multiscale version (without the fourth item) of Theorem 3, since w* 
takes its values in the interior of the constraint set [5/2, +oo), it follows in particular that 

f) M 

— {p*{s),x*{s),Pp{s),p,^{s),p°,w*{s),u*''^°P\s)) = 0, 

for almost every s G [0,1). Moreover, since / 9 *( 1 ) = b* belongs to the interior of (b* — 5, b* +5), 
we can select (pp,Px) such that Pp{l) = 0 (usual transversality condition on the adjoint vector). It 
follows that 

{Vx{s),f{p*{s),x*{s),u*''^°P\s))s,r. +p°f{p*{s),x*{s),u*^‘^°P''{s)) = -Pp{s), 

for almost every s G [0,1]. We have thus proved that the function t i-J- H{t,q*{t),p{t),p^,u*'^{t)) 
coincides, almost everywhere on [b* — 5,b*), with a continuous function vanishing at t = b*. 

Remark 31. Another method in order to prove the fourth item of Theorem 3 consists of consider¬ 
ing variations of the final time 6 in a neighborhood of b* and to modify accordingly the functional 
of Section 4.3.2 to which Ekeland’s variational principle is applied. 

4.4 Proof of Theorem 2 

The following proof can be read independently of the rest of Section 4. It is inspired from [50, 
Chap. 6]. We only treat the case where the final time b G T is fixed. The proof can be easily 
adapted to the case of a free final time. 

Let us consider a sequence {qk)kGN of AI, associated with sampled-data controls Uk G (Ti, fl), 
minimizing the cost considered in (OSDCP)j^. It follows from the assumptions that the sequence 
(/(•, qk, uf), /°(-, qk, uf))k(£tq is bounded in L^([a, 6 )t, K"+^). Hence a subsequence (that we do not 
relabel) converges in the weak topology of L^([a, b)j, IR"+^) to a function {F, F^). Moreover, a sub¬ 
sequence of (gfc(a))fcgN (that we do not relabel) converges in M" and we denote the limit by x{a). We 
define the absolutely continuous functions x{t) = x{a) + F{t)At and x°(t) = F°(r)Ar. 
In particular, note the pointwise convergence of {qk)keN to x on [a, 6]t. Since g is continuous and 
since S is closed, we have g{x{a),x{h)) G S. Note that x°( 6 ) is equal to the infimum of admis¬ 
sible costs. To conclude the proof, we have to prove the existence of u G L5P(Ti,n) such that 
x{t) = x{a) + f{T, x(t), u^{t))At and x°(t) = /°(r, x(r), u‘^(r))AT. 

The sequence (/(•, x, rtf'), /°(-, x, uf ))fcgN is bounded in L|([a, 6)t, K"+^). Hence, a subsequence 
(that we do not relabel) converges in the weak topology of L|([a, 5)t, M"+^) to some {F,F^). It 
follows from the Lebesgue dominated convergence theorem and from the global Lipschitz continuity 
of / in 9 on [a, 6 ]t x Br,.(0,M) x fl that v?(r)F(r)AT for every (p G 

L^([a, 5)t, ffi). Hence F = F (and similarly = F^) A-a.e. on [a, 6)t. 

We define the set W of functions 0 G L|([a, 5)t, M"+^) such that 0(<) G W(t,x(t)) for A-a.e. 
t G [a, 6)t. It follows from the assumptions that W is a closed (convex) subset of L^([a, 6)t, 
with its usual topology, and thus with its weak topology as well (see [19]). We infer that (F, F°) = 
(F, F°) G W. Hence, for A-a.e. t G [a, 6)t, there exists v(t) G H such that (F(t),F°(t)) = 
{f{t,x{t),v{t)),f°{t,x{t),v{t))). Note that v can be selected /r a- measurable on [ 0 , 6 ) 1 -^ 

Since RSi n [a,5)T is at most countable, using a diagonal argument, there exists a subsequence 
of (uk)keN (that we do not relabel) such that Uk(r) converges to some Ur G fl for every r G 
RSi n [a, 5 )t. It follows from the Lebesgue dominated convergence theorem that x(t) = x(a) + 

^Indeed, similarly to the proof of Proposition 1, one has just to consider the extension function v defined on 
[a, b) by v(t) = v(t) if t E T and by v(t) = v(r) if t G (r, cr(r)), where r E RS. Then, the function v can be selected 
/ij^-measurable on [a, 6) from [43, Lemma 3A p. 161], which implies the p^-measurability of v on [a,b)j from [23]. 
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/[a.t)T and x°{t) = /0(r, x{t),u'^{t))At, where u is defined by u{t) = Ur 

if t = r e RSin[a, 6)t and u{t) = v{t) if f G RDin[a, &)t- It is easy to prove that u € Lip ([a, 6)ti, fl) 
(the /tai -measurability of u on [a, 6)ti is established as in the proof of Proposition 1). This 
concludes the proof of Theorem 2. 

Acknowledgment. The second author was partially supported by the Grant FA9550-14-1-0214 
of the BOARD-AFOSR. 
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